- Description
- A ship launch
- Source
- author not stated · original page
- Licence
- PD Public domain
Deployment and Operations
This topic measures the safety of production release and day-to-day operations: deployment duration and reversibility, change failure rate, operational autonomy…
In brief
This topic measures the safety of production release and day-to-day operations: deployment duration and reversibility, change failure rate, operational autonomy, and control of environments and configuration. It extends the chain seen in 05 (producing the artifact) with the chain “put it into service and keep it running”.
Level: 🟢→🔴 · Topic code: DEP (DL matrix)
Why this topic#
A deployment is the moment when value meets risk. A manual, overnight deployment lasting several hours, with no prepared rollback, creates incidents that stay in collective memory and eventually slow down all deliveries (“we're not touching anything before the end of the quarter”). A normalized deployment — short, repeatable, reversible — makes delivery boring, which is exactly the goal.
Two specifics of our context: (1) the database is not deployed like a binary: it carries state, and a migration is often irreversible (column deletion, data transformation); (2) when the product is delivered to the client via installation kits, the client executes the deployment: the promise becomes “a kit installs without surprises, regardless of the starting version.” See CI/CD for Data: Databases, Datamarts, Pipelines, and Reports and topic BI 07.
What this topic measures#
- Axis: from “heroic intervention” to “routine, reversible, and measured event.”
- Key indicators: downtime duration per deployment; change failure rate; time to restore service; share of deployments with no manual intervention.
- Unit: minutes; %; hours.
Five-level table#
| Level | Short name | Promise | Typical indicator (orders of magnitude) |
|---|---|---|---|
| 1 | Heroic | Production release succeeds if the right people are present and available. | Deployment lasting several hours; frequent failures (> 30%) |
| 2 | Procedure-driven | A deployment follows a written procedure and leaves a trace. | Procedure; rollback by restore; failure ≈ 20-30% |
| 3 | Repeatable | A deployment is the same operation everywhere, in under an hour, with a tested rollback. | Scripted; tested rollback; failure < 15% |
| 4 | No interruption | The service stays available during deployment, and a defect affects few users before being contained. | Downtime ≈ 0; progressive deployment; restoration < 1 h |
| 5 | Autonomous | The system deploys, verifies, and corrects itself; production release is a non-event. | Failure < 5%; automatic rollback; restoration in minutes |
Milestones by level#
| ID | Milestone | Verifiable evidence |
|---|---|---|
| DL-DEP-1.1 | The list of environments and their roles exists | Up-to-date page |
| DL-DEP-1.2 | Deployments are logged (who, when, which version) | Consultable log |
| DL-DEP-1.3 | Pre-deployment backups exist | Backup evidence per deployment |
| DL-DEP-2.1 | A deployment procedure and a rollback procedure are documented | Dated documents |
| DL-DEP-2.2 | Environment-specific parameters are listed (no “magic” values in code) | Configuration inventory |
| DL-DEP-2.3 | Deployment permissions are separated from development permissions | Role review |
| DL-DEP-2.4 | Deployment is announced and scheduled in a maintenance window | Change calendar |
| DL-DEP-3.1 | Deployment is end-to-end scripted, with no manual step | Successful execution by a third party |
| DL-DEP-3.2 | Rollback is tested, at least once per quarter | Dated exercise |
| DL-DEP-3.3 | Database migrations are versioned, idempotent, and replayable (rerun does not break) | Successful double execution |
| DL-DEP-3.4 | Environments are described as code (or reconstructable) and drift little | Periodic drift check |
| DL-DEP-3.5 | An automatic smoke test verifies each deployment | Logged result |
| DL-DEP-4.1 | A deployment causes no downtime (blue/green, rolling, compatibility window) | Downtime measure ≈ 0 |
| DL-DEP-4.2 | New features can be enabled by toggle, independently of deployment | At least 3 toggles used |
| DL-DEP-4.3 | A progressive deployment (canary, rings) limits defect exposure | Configurable exposure percentage |
| DL-DEP-4.4 | Data migrations are decoupled (expand/contract) and N-1 compatible | Tested procedure |
| DL-DEP-4.5 | Change failure rate is measured and < 15% | Quarterly report |
| DL-DEP-5.1 | Rollback is automatic when a health threshold is exceeded | Test triggering rollback |
| DL-DEP-5.2 | Changes are analyzed after each failure (blameless analysis) and actions are tracked | Closed incident reports |
| DL-DEP-5.3 | Failure exercises (disaster recovery, loss of an environment) are conducted at least annually | Exercise report |
| DL-DEP-5.4 | Operations cost per deployment and per environment is known and optimized | Cost table |
Detailed reading by level#
🟢 Level 1 — Heroic#
- Signs: overnight deployments, manual file copying, each version is an event, incidents are handled “hot” by the only person who knows how.
- Gain: individual mastery.
- Does not guarantee: repeatability or availability of key people.
- Difficulty: 1/5 · Effort: 1/5.
🟢 Level 2 — Procedure-driven#
- Signs: a checklist, a log, backups. Deployments are slower but less risky.
- Gain: knowledge is documented.
- Does not guarantee: that the procedure is followed to the letter — human factors remain.
- Difficulty: 2/5 · Effort: 2/5.
🟡 Level 3 — Repeatable#
- Signs: deployment by script; rollback is tested (a rollback never replayed is a hypothesis). Database migrations are idempotent: running the script twice leaves the same state, which allows recovery after a mid-run failure.
- Gain: regular, boring deployments; fewer production release incidents.
- Does not guarantee: continuous availability — brief downtime still exists.
- Difficulty: 3/5 · Effort: 3/5.
🔴 Level 4 — No interruption#
- Signs: deployment without service stop, feature toggles, progressive exposure. Schema migration is done in two steps: add (expand), migrate, then remove (contract) once no consumer uses the old one anymore.
- Gain: deployment is no longer correlated with business risk.
- Does not guarantee: absence of incidents; it limits and contains them.
- Difficulty: 4/5 · Effort: 4/5.
🔴 Level 5 — Autonomous#
- Signs: automatic rollback on health-indicator degradation; blameless analysis; regular failure exercises.
- Gain: proven, not assumed, resilience.
- Does not guarantee: profitability — advanced automation has a design cost that only makes sense beyond a certain change volume.
- Difficulty: 5/5 · Effort: 5/5.
Partial achievement and rating#
Apply the rating rule.
- 0.5 on DL-DEP-3.2: rollback tested once, more than one year ago.
- 0.5 on DL-DEP-3.3: idempotent migrations for half of objects (the others fail on the second pass).
- A partial 5: automatic rollback for the application, manual for the database.
Illusory rollback. Symptom: “we'll restore the backup” — but since deployment, users have entered data. Cause: confusing restore with rollback. Remedy: N-1 compatible changes so returning to previous code does not require restoring data.
Environment drift. Symptom: “it won't install at this client.” Cause: manual changes. Remedy: describe as code and verify drift.
Test your installation procedure on a clean environment and on an environment at the previous version: those are two different paths; testing only one means missing the other.
Moving to the next level#
Possible means (optional)
- 1 → 2: checklist, deployment log, mandatory backup beforehand.
- 2 → 3: single script, smoke test, idempotent migrations, rollback exercise.
- 3 → 4: blue/green or progressive deployment, toggles, two-step migrations.
- 4 → 5: health monitoring tied to automatic rollback; failure exercises.
A migration removes a column used by two reports. Level 2: direct removal, reports break Monday morning. Level 4: (1) add the new column, (2) migrate consumers over 2 versions, (3) remove the old one in the next version, with a compatibility view in the meantime.
A “zero deployment incidents” objective pushes teams to declare incidents as “normal changes.” Also measure deployments delayed or avoided out of fear.
See also#
- Neighboring topics: 05 CI · 07 Security · 08 Observability and Reliability · 09 Technical Debt and Maintainability.
- Other matrices: Project — delivery and production release · Data — operations and support · BI — operations, monitoring, and capacity.
- Practice: CI/CD · CI/CD for Data: Databases, Datamarts, Pipelines, and Reports · Azure DevOps / TFS · Version control system and SQL/BI data.
- Glossary · Rating.
✻ Written by Claude Sonnet 5.5
