i

Description

Deux poids en laiton couverts de poinçons de vérification : la conformité attestée avant l'usage

Source

HelgeRieder · original page

Licence

CC0 Public domain

Original photo, unretouched. The background is a version of it stylised by the site.

Acceptance testing and business validation

“The code runs without errors” does not mean “the number is correct.” In data, acceptance testing has a specific nature: you don't verify a function, you verify a number against an…

On this page

1. Why this theme
2. What this theme measures
3. The five levels
4. Milestones by level
5. Detailed reading by level
   1. 🟢 Level 1 — Impression
   2. 🟢 Level 2 — Sample
   3. 🟡 Level 3 — Plan
   4. 🟡 Level 4 — Automated
   5. 🔴 Level 5 — Continuous
6. Partial achievement and rating
7. Moving to the next level
8. See also

Level: 🟢→🔴 · Theme code: REC

Why this theme#

“The code runs without errors” does not mean “the number is correct.” In data, acceptance testing has a specific nature: you don't verify a function, you verify a number against an independent reference (an accounting statement, an export from the source application, an expert's manual calculation). Without it, trust is based on the absence of complaints. With it, you can say: “this number matches the reference within 0.1%; here is the explained and signed-off gap.” It is the last line of defense before a decision. This theme is distinct from code testing (DL — Testing), model and measure testing (BI — Testing and Metrics Reliability), and quality (QUA): here, it's about business validation on numbers, its evidence, and its sign-off.

What this theme measures#

• Axis: from a feeling of correctness to continuous reconciliation against independent references.
• Key indicator: share of critical indicators reconciled with an independent reference, and explained residual gap.
• Unit: % of reconciled indicators; maximum tolerated relative gap (e.g., 0.1%); number of escaped defects detected in production.

The five levels#

Milestones by level#

Detailed reading by level#

🟢 Level 1 — Impression#

• Signs: “looks right to me”; no record of what was checked; the right number is the one that looks like the previous one.
• Gain: fast social validation.
• Does not guarantee: accuracy; a 2% error goes unnoticed.
• Difficulty: 1/5 · Effort: 1/5.

🟢 Level 2 — Sample#

• Signs: a table comparing five report figures to those from the source application; gaps are discussed; the file is archived.
• Gain: most scope or join errors are spotted.
• Does not guarantee: coverage or repeatability.
• Difficulty: 2/5 · Effort: 2/5.

🟡 Level 3 — Plan#

• Signs: versioned plan; edge cases listed; business sign-off; consistency across reports tested.
• Gain: reproducible acceptance testing, shared responsibility with the business.
• Does not guarantee: no regression in the next version (manual acceptance testing, therefore expensive to replay).
• Difficulty: 3/5 · Effort: 3/5.

🟡 Level 4 — Automated#

• Signs: a reference figure set runs in the pipeline; a “before/after” result diff is produced for each version; each incident adds a case.
• Gain: acceptance testing that costs almost nothing to replay; regressions become visible.
• Does not guarantee: that the reference is correct (an incorrect reference locks in the error).
• Difficulty: 4/5 · Effort: 4/5.

-- Réconciliation : total du datamart contre la référence source (T-SQL), tolérance 0,1 %
WITH Dm  AS (SELECT SUM(REL_QTE) AS Tot FROM [fact].[Releve] WITH (NOLOCK) WHERE [date] >= '2026-09-01' AND [date] < '2026-10-01'),
     Ref AS (SELECT SUM(QTE)     AS Tot FROM [ref].[ExportApplication] WITH (NOLOCK) WHERE MOIS = '2026-09')
SELECT Dm.Tot AS Datamart, Ref.Tot AS Reference,
       ABS(Dm.Tot - Ref.Tot) * 1.0 / NULLIF(Ref.Tot, 0) AS EcartRelatif,
       CASE WHEN ABS(Dm.Tot - Ref.Tot) * 1.0 / NULLIF(Ref.Tot, 0) <= 0.001 THEN 'OK' ELSE 'ECHEC' END AS Verdict
FROM Dm CROSS JOIN Ref;

🔴 Level 5 — Continuous#

• Signs: reconciliation runs daily; a gap alerts an owner; coverage is measured by indicator.
• Gain: trust is built continuously, not at delivery time.
• Does not guarantee: meaning (perfect reconciliation of a poorly defined indicator does not make it useful; see VAL).
• Difficulty: 5/5 · Effort: 4/5.

Partial achievement and rating#

Assess based on the five most used indicators. A milestone is worth 1 if all five are covered, 0.5 if two to four, otherwise 0. Without archived evidence, an acceptance milestone is worth 0: acceptance testing without proof did not happen.

A “partial 5”: daily reconciliation covers revenue and main volumes (5.1 at 0.5), alerts reach the owner (5.2 at 1), references are stable but not tracked (5.3 at 0.5), coverage is not measured (5.4 at 0).

Moving to the next level#

Possible means (optional)

Cross-cutting pitfalls: (1) False positive — 100% success on a non-representative test set. (2) Goodhart — targeting “zero gap” leads to adjusting the reference or widening tolerance; track every tolerance change. (3) Tunnel effect — acceptance testing that is only technical, without a business decision-maker, lets definition errors through (GOV).

See also#

• Same matrix: 03 Data Quality · 06 DataOps · 09 Adoption · 10 Operations · README
• Other matrices: PI — quality and acceptance testing · DL — Testing · BI — Testing and Metrics Reliability · BI — DAX Measures and Calculations
• Practice: methods specific to data projects · CI/CD for data · reporting project checklist
• Glossary · Rating

✻ Written by Claude Sonnet 5.5