Skip to content
A second look at the obvious answer
Think Twice TodayA second look at the obvious answer

Evidence

A single story is evidence of possibility, not of frequency

An anecdote establishes that something can happen and says almost nothing about how often it does, which is precisely the inference people draw from it anyway.

By Gautam Pillai4 min read

Editorial note. Independent reporting and analysis. Nothing here is sponsored or paid for. How we work.

What a story genuinely establishes

A single case isn’t worthless evidence, and the dismissive treatment it often receives is too crude. One well-documented instance proves that the thing occurs, that the mechanism can operate, and that a claimed impossibility is false. Refuting a universal statement takes exactly one counterexample, and in that role an anecdote is decisive rather than weak.

It also does something no summary statistic does: it shows what the phenomenon looks like in enough detail to generate hypotheses. Case reports are how a great many effects were first noticed, and treating them as beneath consideration would have delayed a substantial amount of useful knowledge. The problem isn’t the story. It is the inference people make from it.

The inference it can’t support

From one case you can conclude that the rate is above zero. You can’t conclude anything about whether it’s common, whether it is more common than the alternative, or whether the thing that appeared to cause it did. All of those require a denominator, and a story arrives without one — you are told what happened to the person in front of you and nothing at all about the people it did not happen to.

The trouble is that the missing denominator is not experienced as missing. The story is complete as a story, it has a beginning and an ending, and the mind reads completeness as sufficiency. Nobody finishes an anecdote with the feeling that a crucial piece of information was withheld, which is exactly why the inference goes through.

Why stories persuade so much better than rates

Several things stack up. A story supplies a concrete image, which the mind uses to judge frequency, so a vivid case raises the estimated rate directly. It supplies a causal narrative in which each step follows from the last, and coherence is read as evidence. And it comes with a person, which engages attention in a way a proportion never will.

None of this is a trick used by dishonest people. It is how communication about anything has always worked, and a presentation with no concrete instance in it is generally understood poorly and remembered worse. The reasonable response is not to distrust stories but to notice which of their effects on you is legitimate and which is arithmetic that never happened.

How the story reached you is the important question

Cases arrive by selection, always. Somebody chose to tell this one, and the reasons a case gets told are strongly related to being unusual, dramatic, or supportive of a point the teller wanted to make. That selection is invisible in the story itself and it is the single most important fact about it.

This is why a collection of anecdotes is not a substitute for a study, however large the collection. Ten stories assembled by the same filter are ten draws from the same biased sample, and the agreement between them measures the filter rather than the world. A hundred testimonials on a page tell you what a hundred people were willing to say and nothing about how many people were asked.

Using stories properly rather than banning them

The honest arrangement is to let the anecdote do the jobs it can do and to attach the rate separately. The story shows what the thing looks like and why it matters; the number says how often it happens. Each without the other is incomplete in a different direction, since a bare rate is under-weighted just as reliably as a bare story is over-weighted.

A useful habit when a case is offered as an argument is to ask what the teller expects the frequency to be, out loud. Frequently the answer reveals that no claim about frequency was intended, at which point the disagreement dissolves. Sometimes the answer reveals a very strong claim about frequency resting entirely on the one case, which is worth knowing before the conversation continues.

Your own experience is an anecdote too

The hardest version to apply is the personal one. What happened to you is a single case, selected by the fact that you were there, remembered by a system that keeps the memorable, and interpreted by somebody with a stake in the interpretation. It carries the full force of first-hand knowledge and roughly the evidential weight of any other single case.

This does not mean discounting your experience, which would be both impossible and unwise, since it is often the only information you have about your own situation. It means holding it as one data point with a known collection bias, and being especially careful when it disagrees with a much larger body of evidence. The most likely explanation for such a disagreement is not that the larger body is wrong. It is that you are one case and it is many.

Common questions

Is an anecdote ever decisive?

Yes, against a universal claim. If somebody asserts that a thing never happens or always happens, a single well-documented counterexample settles it. That is a narrow but genuine role, and it is the one situation where a story outweighs a statistic.

What about a large number of similar reports?

It depends entirely on how they were collected. Reports gathered by an active surveillance system that seeks out all cases are informative; reports that arrived because people chose to submit them measure willingness to report as much as they measure occurrence.

How should I present my own experience in an argument?

As what it is — an existence proof and an illustration, offered with an explicit note that you do not know how typical it is. Stating that limit yourself is more persuasive than having it pointed out, and it keeps the useful part of the story in play.

Evidenceanecdoteevidencepersuasionfrequency
Gautam Pillai
Reporter, Think Twice Today

Gautam writes about biases, choices, risk, mostly the parts other people skip and reads the small print so you do not have to.