The famous curve about incompetence may be mostly a measurement artefact

What the popular version says

The claim as it circulates is that people who are bad at something believe they are good at it, while genuinely capable people underrate themselves. It is repeated constantly, usually as an explanation of somebody else, and it has the shape of a satisfying reversal, which is generally a warning sign about how carefully a finding is being reported.

What the original research actually reported is more modest. When people are sorted by test performance and then asked to estimate their own percentile, the lowest scorers overestimate substantially and the highest scorers underestimate slightly. Notably, the low scorers still rate themselves below the high scorers on average. Nobody in the data believed they were the best.

The interpretation and the objection to it

The interpretation offered was metacognitive: the skills needed to perform well are the same skills needed to evaluate performance, so incompetence is partly an inability to recognise incompetence. It is an elegant story and it may contain something true.

The objection is that the pattern in the graph is what you would expect even if that story were entirely false. Self-estimates are noisy and only loosely related to actual scores, and everybody tends to rate themselves as somewhat above average. Combine a weak correlation with a general upward tilt and you produce exactly this shape: the bottom group overestimating, the top group underestimating, with the crossover somewhere in the upper middle.

Why the artefact explanation is hard to dismiss

Several analyses have demonstrated the pattern using simulated data in which self-assessment carries no information about skill whatsoever. If random self-estimates reproduce the graph, then the graph on its own cannot be evidence for the metacognitive account. This is closely related to the general fact that extreme measured values are followed by less extreme ones, which is a property of imperfect measurement rather than a psychological force.

That is not a proof that the original interpretation is wrong. It is a proof that the most-cited evidence for it does not discriminate between the interesting hypothesis and a dull one, which is a different and more common situation than a finding being refuted.

What survives the argument

Two things survive, and they are enough to be going on with. First, self-assessments of ability correlate weakly with measured ability across a wide range of tasks — the association is positive, and it is much weaker than people assume when they rely on their own sense of how they are doing. Second, people on average rate themselves above average on most desirable traits, which cannot be true of everyone.

Both of those are well documented and neither requires the specific metacognitive story. Together they support the practical conclusion that people usually reach anyway, which is that your own estimate of your competence is a poor instrument. The famous curve is not needed to get there.

Why self-assessment is hard even for careful people

The structural problem is a lack of comparison. To place yourself in a distribution you need to know the distribution, and most people see a heavily filtered sample of other people’s work — the visible, the published, the presented — with the ordinary middle almost entirely absent. That distorts the reference set in both directions depending on which slice you happen to see.

There is also the matter of what you are estimating. Ability at a task is not one quantity, and people asked how good they are tend to answer about the parts they can see themselves doing. The parts you cannot see are precisely the ones that would revise the estimate.

Feedback rarely closes the gap either, because most of the feedback available to an adult is either absent, delayed or softened by whoever is delivering it. A person can spend years doing something without ever being told plainly where their work sits relative to anybody else’s, and in that vacuum the estimate is built from whatever comparisons happen to be visible. That is a condition of the environment rather than a defect of the person, which is why it responds to arrangements and not to resolutions.

What actually improves the estimate

Anything that replaces self-judgement with a comparison against something external. A task with a scored answer, a sample of work from people at a known level, a measurable outcome that arrives regardless of interpretation. These are unglamorous and they do the job the introspective estimate cannot.

And when the effect is invoked in conversation, notice what it is being used for. Almost always it is applied to somebody else, which is the one direction in which it carries no information at all. The version worth keeping is that nobody, including the person citing the study, has reliable access to their own place in a distribution.