i

Description

Un petit ordinateur d'échecs électronique en pleine partie, pièces blanches et noires sur l'échiquier.

Source

Dezidor · original page

Licence

PD Public domain

Original photo, unretouched. The background is a version of it stylised by the site.

Hofstadter’s Law

Even when we know delays are natural and we build in buffer, we still end up late.

In brief

• What : a complex task always takes longer than expected, even if this rule has been taken into account.
• Why : self-reference: the correction we apply is itself estimated with the same optimism as the original estimate.
• Detect : “we already added buffer,” and yet the delay still happens.
• Avoid : adjust using your data, not your impression; break work down, deliver early, and revise estimates along the way.

On this page

1. Definition
2. The mechanism
3. Where you encounter it
   1. In project management
   2. In management
4. Famous cases
   1. “In ten years, a computer will be world chess champion” 1957-1997
   2. The 90-90 rule and Brooks’s law 1975-1985
5. Detection signals
6. Avoidance reflexes
7. With AI: detect, fix, mitigate
   1. Detect
   2. Fix
   3. Mitigate
   4. Prompts to integrate into your workflow
   5. How to integrate them
   6. Precautions
8. Limitations
9. Sources and verification
10. See also
    1. Combines with, or reinforces
    2. Related biases
    3. Counterweights and related reflexes

Definition#

Fictitious illustration

Hofstadter’s law is stated as follows: “It always takes longer than expected, even when taking Hofstadter’s law into account.” It is due to Douglas Hofstadter, who formulated it in Gödel, Escher, Bach: an Eternal Golden Braid (1979), regarding predictions of when a computer would defeat the best human chess players.

It is an empirical, intentionally paradoxical adage, not a cognitive bias in the strict sense nor a proven scientific law. Its strength is its self-reference: it is true because it applies to itself. It reminds us that we cannot solve the problem by adding an “intuitive” buffer, since the intuition setting that buffer is the same one that got it wrong in the first place. It is close to The Planning Fallacy, whose persistent effect it describes, and to Unrealistic optimism, of which it describes one of the drivers.

The mechanism#

No experiment has “proven” the law; however, we can explain why it is often observed:

• Recursion of optimism : we adjust our estimate (“let’s add 20%”), but the adjustment is itself decided with the same optimism that produced the estimate. The correction remains too small.
• Unexpected issues cannot be imagined in advance : we add buffer for known risks, not for those we cannot yet see.
• The distribution tail is heavy : a task’s actual duration is not symmetric around the average; a few very large overruns pull the average upward, and a buffer set for a “reasonable overrun” does not cover them.
• Effect of the buffer itself : a known buffer may get consumed (Parkinson’s law: work expands to fill the available time), so it is gone by the end.
• Misleading proportion : we estimate the visible part of a task; the final share of work (integration, testing, finishing) takes far more time than the beginning suggests. This is the “90-90 rule” attributed to Tom Cargill of Bell Labs: the first 90 percent of the code takes 90% of the time, and the last 10 percent takes the remaining 90%.

Where you encounter it#

In project management#

• Estimating : “I already added buffer”; the buffer itself is underestimated compared with historical data.
• Release planning : the date is pushed back three times, each time by “two weeks,” never by what observed velocity indicates.
• The last 10 percent : the project has been “almost done” for weeks; the ending (acceptance testing, fixes, production rollout) is what takes the most time.
• Story-point estimation : velocity is calibrated on past sprints, but commitments are made using the best observed velocity, not the median.
• External dependencies : we add buffer for our own work, not for waiting on a client decision or a third-party delivery.

In management#

• Commitments to leadership : a date is given “with buffer,” it still has to be postponed, and trust erodes.
• Transformation plans : the adoption time for a tool or method is revised upward at each milestone.
• Delegation : you ask a team member “how much time they need,” add what you consider reasonable buffer, and are surprised by delay.
• Hiring : the timeline “to find and onboard someone” gets longer whenever the time to true autonomy is not accounted for.

Famous cases#

Detection signals#

• “We already added buffer.” And we still overrun.
• The date has been pushed back several times, each time by the same duration (“two more weeks”).
• Progress has been “90%” for several weeks.
• The buffer is backed by no data: “let’s take 20%,” “let’s double it.”
• No one knows what overrun percentage the team had on its last three projects.
• A task’s buffer is consumed right away (work expands), and delay accumulates at the end.

Avoidance reflexes#

1. Adjust with data, not a rule of thumb. Calculate the actual/estimated ratio from your latest projects (median and 80^th percentile) and apply it, instead of adding “a little buffer.”
2. Revise estimates as you go. After each milestone, recalculate the finish based on observed velocity, not hoped-for velocity.
3. Deliver early and in small chunks. A short batch is easier to estimate; delay becomes visible from the first batch rather than at the end.
4. Estimate the ending separately. Reserve an explicit line item for integration, acceptance testing, bug fixing, and production rollout.
5. Announce a range and a confidence level. “Between 10 and 16 weeks, 80% confidence” is better than a single date with invisible buffer applied.
6. Reduce scope rather than quality. When delay appears, cut features, not acceptance testing or tests.
7. Place buffer at the project level, not the task level. A global reserve managed by the lead is less likely to be consumed than hidden buffer in each task.

With AI: detect, fix, mitigate#

Prompts to integrate into your workflow#

1. Correct the estimate with historical data. Use with an export of your estimates and actual durations.

Here is the history of our past work packages (initial estimate, actual duration, package type): [table].
1. Calculate, for each package type, the actual duration / estimated duration ratio: minimum, median, 80<sup>th</sup> percentile, maximum.
2. Say which conclusions are reliable and which are based on too few packages.
3. For a new package estimated at [value], give the duration to announce for “a one-in-two chance” and for “four chances out of five.”
Do not invent any data: if the history is insufficient, say so.

2. Revise the date during the project. At each milestone, with the actually observed figures.

Situation: [remaining work in units], observed velocity over the last [N] sprints: [values], current delivery date: [date].
Calculate a likely end date with a range (optimistic, likely, pessimistic) based on <em>observed velocity</em>, not expected velocity.
Specify what would make this projection too optimistic (project closeout, external dependencies, fixes) and suggest three scope-reduction decisions to meet the date if it is mandatory.

How to integrate them#

• During estimation: prompt 1 is used to correct an estimate with historical data before announcing it.
• At each milestone: prompt 2 replaces “we'll catch up” with a quantified projection.
• In retrospectives: keep the results to enrich the history: it's that history, not AI, that corrects the bias.

Precautions#

• A history that is too short or biased (only completed projects) gives misleading corrections: include abandoned projects.
• AI can produce incorrect calculations confidently: verify ratios on a sample.
• Do not present an AI calculation to a client as a commitment: it's a decision aid.
• Do not send histories containing confidential data to an unauthorized tool.

Limitations#

• Hofstadter's law is not a cognitive bias in the strict sense: it's an empirical law, formulated with humor, that summarizes a common experience. It was not established through controlled experiments and has no quantified formulation.
• It is unfalsifiable by design: for any estimate, whatever it is, one can say the law was already included. It is useful as a warning, not as a testable prediction.
• It does not apply to every task: for repetitive, well-measured, low-uncertainty activity, estimates are often accurate. It targets complex and new tasks.
• Not all projects run over: some finish early or on time; overruns, being more pronounced, are what pull the average.
• The “90-90 rule” and “Brooks's law” are professional wisdom, useful but not proven in a lab.

Sources and verification#

• Hofstadter (1979), Gödel, Escher, Bach: An Eternal Golden Braid, Basic Books — statement of the law; Pulitzer Prize 1980 (General Nonfiction category).
• Simon & Newell (1958), “Heuristic Problem Solving: The Next Advance in Operations Research,” Operations Research — the four predictions from 1957.
• IBM, “Deep Blue” — May 1997 match against Kasparov.
• Brooks (1975), The Mythical Man-Month, Addison-Wesley — programmer optimism, Brooks's law.
• Bentley (1985), “Programming Pearls: Bumper-Sticker Computer Science,” Communications of the ACM — the 90-90 rule, attributed to Tom Cargill.
• Buehler, Griffin & Ross (1994), “Exploring the planning fallacy,” Journal of Personality and Social Psychology — the experimental basis of the phenomenon.
• Parkinson (1955), “Parkinson's Law,” The Economist, November 19, 1955.

See also#