Matrix: Data governance

How the product turns its data into a managed asset: measured quality, metadata, catalog, lineage, owners, personal data, classification, anomalies.

≈ 11 min readReading

In brief

8 dimensions, levels 0 to 5, 112 criteria.
Tool: self-assessment, choosing "Data governance".
Levels and criteria: original synthesis, references cited at the bottom of the page.

On this page

Overview
Quality
1. Measured quality
2. Data anomaly management
Knowledge of the data
Responsibilities and compliance
Sources and verification

Scope

What the product puts in place to govern the data it holds: quality measurement, metadata and glossary, catalog, lineage, owners, compliance with personal data rules, classification, handling of anomalies. Governance of the organization as a whole falls under the DAMA-DMBOK matrix, which describes an organization, not a product.

8 dimensions, six levels (0 to 5) and 112 observable criteria. To place your product: open the self-assessment for this theme. The calculation rules are here.

Overview#

Each cell is the level's signature for the dimension.

Dimension	0 · Handcrafted	1 · Locally tooled	2 · Shared	3 · Industrialized	4 · Integrated and governed	5 · Autonomous
Measured quality	Quality is judged by feel, when a user complains.	Known defects.	Defined rules.	Quality tracked.	Quality managed.	Quality anticipated.
Metadata and glossary	The meaning of the data lives in the heads of a few people.	Scattered notes.	Data dictionary.	Metadata linked to objects.	Metadata put to use.	Living metadata.
Data catalog	You find data by asking the right person.	List of datasets.	Searchable catalog.	Up-to-date catalog.	Connected catalog.	Active catalog.
Traceability and lineage	The origin of a figure is reconstructed by reading the code.	Manual diagrams.	Declared dependencies.	Table lineage.	Column lineage.	Standardized lineage.
Owners and responsibilities	No one answers for the data: everyone touches it.	Known contacts.	Designated owners.	Responsibilities exercised.	Governance body.	Responsibilities supported by tooling.
Personal data and register	Personal data is not identified.	Data identified.	Register.	Individuals' rights.	Built-in protection.	Compliance checked.
Classification and policies	All data is treated the same way.	Sensitive data spotted.	Classes defined.	Rules applied.	Policies under control.	Automated classification.
Data anomaly management	Anomalies are fixed on request, with no trace.	Anomalies noted.	Handling process.	Fix at the source.	Anomalies analyzed.	Anomalies prevented.

Link with the global model

Dimensions of the global model most concerned: Data quality, Documentation and traceability and Security and compliance.

Quality#

Measured quality#

Is data quality defined, measured and tracked over time?

Level	Signature	Criteria to meet
0 · Handcrafted	Quality is judged by feel, when a user complains.	Starting point: no criteria.
1 · Locally tooled	Known defects.	Known data defects are listed. Each defect has someone responsible for fixing it.
2 · Shared	Defined rules.	Quality rules are defined for the main datasets (completeness, uniqueness, validity, consistency). These rules are run on every load. The result of each rule is kept with its date.
3 · Industrialized	Quality tracked.	A quality indicator per dataset is calculated and tracked over time. An acceptability threshold is set for each rule, with the associated decision (block, flag). The rules apply to the sources as well as to the published data.
4 · Integrated and governed	Quality managed.	The quality rules are written with the business owners. The quality indicator is visible to users alongside the data. The causes of the most frequent defects are analyzed and addressed at the source.
5 · Autonomous	Quality anticipated.	Anomalies are detected automatically without a rule written in advance (deviation from history, improbable values). A drop in quality is flagged before it reaches the reports. Quality is an acceptance criterion for every data delivery.

Effect on the user

You can say how far a dataset can be trusted, with a number rather than a feeling.

Data anomaly management#

What happens to a data anomaly once it has been detected?

Level	Signature	Criteria to meet
0 · Handcrafted	Anomalies are fixed on request, with no trace.	Starting point: no criteria.
1 · Locally tooled	Anomalies noted.	Reported anomalies are noted. Each anomaly has a person who handles it.
2 · Shared	Handling process.	Anomalies are recorded in a tracking tool, with their severity. An anomaly is qualified: source error, processing error or missing rule. The expected handling time depends on severity.
3 · Industrialized	Fix at the source.	Anomalies caused by the source are sent back to its owner with follow-up. A fix in the product is added only if the source cannot be corrected, and it is documented. Rows kept aside as anomalies are replayed after correction.
4 · Integrated and governed	Anomalies analyzed.	Anomalies are counted by cause, by source and by period. Recurring causes lead to a corrective action decided with their owner. The cost of anomalies (correction time, distorted decisions) is estimated.
5 · Autonomous	Anomalies prevented.	A known anomaly automatically generates a control rule to detect it next time. The correction of simple cases is automated and traced. The number of recurring anomalies decreases from one period to the next, and this is tracked.

Effect on the user

An anomaly is followed through to its correction, and its cause is addressed so that it does not come back.

Knowledge of the data#

Metadata and glossary#

Is the data described, with a shared business vocabulary?

Level	Signature	Criteria to meet
0 · Handcrafted	The meaning of the data lives in the heads of a few people.	Starting point: no criteria.
1 · Locally tooled	Scattered notes.	Notes describe the main fields. Ambiguous business terms are identified.
2 · Shared	Data dictionary.	Each table and each column has a description in a data dictionary. A glossary defines the main business terms. The dictionary and the glossary can be consulted by all users.
3 · Industrialized	Metadata linked to objects.	Descriptions are stored with the database objects (extended properties, column comments), not in a separate document. Each glossary term is linked to the fields that implement it. Changing an object requires updating its description.
4 · Integrated and governed	Metadata put to use.	Metadata feeds the documentation, the models and the report labels. Glossary terms have an owner and a validation date. Metadata is available in several languages.
5 · Autonomous	Living metadata.	Technical metadata (types, volumes, usage) is collected automatically. A missing or outdated description is detected automatically. Terms in conflict between domains are flagged for arbitration.

Effect on the user

A word means the same thing to everyone, and a field is explained where it is used.

Data catalog#

Can you search for and find the available data, with its context?

Level	Signature	Criteria to meet
0 · Handcrafted	You find data by asking the right person.	Starting point: no criteria.
1 · Locally tooled	List of datasets.	A list of the main datasets is kept. The list shows the owner of each dataset.
2 · Shared	Searchable catalog.	A catalog makes it possible to search datasets by name and by keyword. Each entry carries the description, the owner, the freshness and the sensitivity level. The catalog is accessible to all the users concerned.
3 · Industrialized	Up-to-date catalog.	The catalog is fed from the product's metadata, with no double entry. Each entry shows the known reports and uses. Datasets that have been retired or replaced are marked as such.
4 · Integrated and governed	Connected catalog.	The catalog links to the glossary, the lineage and the data contracts. Users can request access to a dataset from the catalog. Certified datasets are distinguished from the others.
5 · Autonomous	Active catalog.	The most useful datasets are suggested according to profile and usage. A dataset with no description or no owner is detected and flagged. Searches with no results are analyzed to complete the catalog.

Effect on the user

Before asking the technical team for data again, you look at what already exists.

Traceability and lineage#

Can you follow a figure back to its source, and find what depends on a piece of data?

Level	Signature	Criteria to meet
0 · Handcrafted	The origin of a figure is reconstructed by reading the code.	Starting point: no criteria.
1 · Locally tooled	Manual diagrams.	A diagram describes the main stages from the source to the reports. The main processes are named in the diagram.
2 · Shared	Declared dependencies.	Each process declares the tables it reads and those it writes. The declaration is kept up to date whenever the process is modified. You can list the tables upstream and downstream of a given table.
3 · Industrialized	Table lineage.	Table-level lineage is calculated from the declared or analyzed dependencies. The lineage includes the reports and models that consume the tables. An impact analysis precedes every structural change.
4 · Integrated and governed	Column lineage.	Lineage is available at column level, with the transformation rules. Each process run records what it read and wrote. Users can consult the lineage from the catalog or the report.
5 · Autonomous	Standardized lineage.	Lineage is collected through an open standard, from all components (scheduling, queries, reports). The cause of a discrepancy in a figure is found automatically by tracing back through the lineage. Breaks in the lineage (undeclared process, unknown source) are detected.

Effect on the user

You can answer "where does this figure come from?" and "what will this change affect?" without an investigation.

Responsibilities and compliance#

Owners and responsibilities#

Does each dataset have someone who answers for it?

Level	Signature	Criteria to meet
0 · Handcrafted	No one answers for the data: everyone touches it.	Starting point: no criteria.
1 · Locally tooled	Known contacts.	For each main dataset, a contact person is known. These contacts are recorded.
2 · Shared	Designated owners.	Each dataset has a business owner and a technical lead. The owner role is described (decisions, validations). The owners are entered in the catalog.
3 · Industrialized	Responsibilities exercised.	Owners validate the definitions, the access rights and the quality rules. An absent owner has a designated deputy. Important decisions about the data are recorded with their date.
4 · Integrated and governed	Governance body.	A committee or a regular ritual brings the owners together to arbitrate. Owners receive a periodic report on the quality and usage of their data. A definition conflict between domains follows an arbitration process.
5 · Autonomous	Responsibilities supported by tooling.	Requests (access, correction, change) are routed automatically to the right owner. A missing or inactive owner is detected automatically. Governance indicators (coverage, lead times) are tracked by domain.

Effect on the user

An anomaly has an identified contact, and a decision about data does not remain without an owner.

Personal data and register#

Do you know which personal data the product processes, why, and how individuals' rights are respected?

Level	Signature	Criteria to meet
0 · Handcrafted	Personal data is not identified.	Starting point: no criteria.
1 · Locally tooled	Data identified.	Columns containing personal data are identified. A person is designated to answer compliance questions.
2 · Shared	Register.	A register of processing activities describes purpose, categories of data, recipients, retention period and security measures. Each addition of personal data is justified by a purpose. The register is kept up to date with every change to the product.
3 · Industrialized	Individuals' rights.	A procedure makes it possible to find all of a person's data in the product. A procedure makes it possible to rectify or erase it, in all layers. Access to personal data is limited to those who need it.
4 · Integrated and governed	Built-in protection.	Personal data is pseudonymized or masked in analysis areas when identity is not needed. An impact assessment is carried out for high-risk processing. Processors and transfers outside the European Union are inventoried.
5 · Autonomous	Compliance checked.	New columns containing personal data are detected automatically and flagged. Retention periods are applied automatically. Individuals' requests are handled through a tooled process, with proof of execution.

Effect on the user

The product can respond to an audit or to a request from an individual without digging through all of its tables.

Classification and policies#

Is the data classified by sensitivity, and do rules follow from it?

Level	Signature	Criteria to meet
0 · Handcrafted	All data is treated the same way.	Starting point: no criteria.
1 · Locally tooled	Sensitive data spotted.	The most sensitive data is identified. The people who have access to it are known.
2 · Shared	Classes defined.	A classification scale (for example public, internal, confidential) is defined. Each main dataset has its class, recorded in the catalog. A handling rule is associated with each class.
3 · Industrialized	Rules applied.	The rules of each class (access, masking, retention, sharing) are enforced by the product. Derived data inherits the highest class of the data it comes from. A change of class follows a decision process.
4 · Integrated and governed	Policies under control.	Policies are published, dated and known to users. Exceptions are recorded with their justification and duration. Compliance with the policies is checked at regular intervals.
5 · Autonomous	Automated classification.	The class is proposed automatically from the content of the data. Sensitive data outside its perimeter is detected and flagged. Policies are applied by executable rules rather than by guidelines.

Effect on the user

Protection measures are proportionate to sensitivity, and applied without case-by-case discussion.

Origin of the criteria

The dimensions are inspired by the references cited below; the levels and criteria are an original synthesis, not a reproduction of a framework. They are offered for discussion and adaptation, not as authoritative.

Sources and verification#

OpenLineage — An open framework for data lineage collection and analysis — open standard for collecting lineage: jobs, runs, datasets; components send their events to a compatible system
CNIL — Le registre des activités de traitement (the register of processing activities) — register provided for by Article 30 of the GDPR: purposes, categories of data, recipients, retention periods, security measures
Bitol — Open Data Contract Standard (ODCS) — a data contract also carries the quality rules, the owners and the security requirements
dbt — Add data tests to your DAG — a data test is a query that returns the failing rows; unique, not_null, accepted_values, relationships
GDPR, Article 5(1)(c) and (e) — text on the CNIL website — data minimisation and storage limitation
DAMA International — DAMA-DMBOK2 — the "Data Governance", "Data Quality" and "Metadata Management" domains; paid framework, therefore not reviewed line by line