Risk and uncertainty are not the same problem

The distinction, and why it was drawn

Economists have long distinguished between situations where the possible outcomes and their probabilities are known, and situations where they are not. The first is risk in the technical sense: a dice game, an insurance pool with a long claims history, a manufacturing process with recorded failure rates. The second is uncertainty, where the probabilities are not merely unknown to you but arguably do not exist as stable quantities at all.

The distinction is old and it has been argued over ever since it was made, particularly by those who hold that any state of belief can be expressed as a probability. That debate is genuine and unresolved. What is not seriously disputed is the practical point: the tools that work when the distribution is known are the wrong tools when it is not, and using them anyway produces an appearance of precision that the underlying knowledge cannot support.

How you can tell which one you are in

The useful test is whether the situation has been repeated enough, under stable enough conditions, to have produced a record. A process that has run ten thousand times under conditions much like today’s supports a frequency, and that frequency can be treated as a probability with reasonable confidence.

A novel situation supports nothing of the kind. A first-of-its-kind project, a market that did not exist five years ago, a regulatory change with no precedent — these can be reasoned about carefully, but the reasoning produces a judgement rather than a measurement. The critical question is whether the past that generated your data was drawn from the same process as the future you are asking about. Where the process itself is changing, historical frequencies describe a world that has gone.

What goes wrong when they are mixed up

The characteristic failure is a model built on historical data, applied to a period when the underlying relationships have shifted, producing outputs that are precise and wrong. The precision is what makes it dangerous, because a number carried to two decimal places is treated as more reliable than a person saying they are not sure, even when the person is better informed.

There is a social dimension to this that is worth naming. Numbers travel through an organisation more easily than caveats do, and by the second or third retelling the qualifications have fallen off while the figure remains. Anyone who has watched an estimate hedged in six ways become a target in a slide two weeks later knows the pattern. The person who produced it did nothing wrong, and the process still ended somewhere indefensible.

A second failure is subtler: the tendency to redefine the question so that it fits the available data. The uncertainty that matters may be whether the whole approach is right, and the model can only speak to variation within the approach. So the analysis addresses the tractable question thoroughly and the important one not at all, and everyone involved comes away feeling that the matter has been examined.

Reasoning usefully under genuine uncertainty

Several methods work under uncertainty even though they produce no probabilities. Identifying which assumptions the plan depends on, and how badly it fails if each is wrong, tells you where to look. Building in the capacity to stop, reverse or change course substitutes flexibility for foresight. Preferring options that perform acceptably across many futures over options that perform superbly in one is a real strategy, not a hedge.

Scenario work is useful for the same reason, provided it is used to test robustness rather than to pick a winner. The value is in discovering that a plan collapses under a scenario nobody had considered, which is information you cannot get from a distribution you do not have. Assigning probabilities to the scenarios usually adds nothing and quietly converts the exercise back into the thing it was meant to replace.

Saying which one you are in

Perhaps the most valuable practice is simply to state, when presenting an estimate, which category it belongs to and on what basis. This figure comes from a long record of similar cases. That one is a considered guess with no comparable precedent. Both are legitimate inputs; they are not the same kind of thing, and a document that presents them in the same typeface has lost the distinction that mattered most.

The temptation runs the other way, because a number looks more professional than an admission. But the cost of a false frequency is paid later and by someone else, which is exactly the structure under which bad practices survive. An estimate with its provenance attached is more useful and more honest, and it invites the right question rather than closing it.