What a control group is actually for

The counterfactual problem

To know whether something worked, you need to know what would have happened without it. That state of affairs never occurred and cannot be observed, which is the fundamental difficulty in all causal inference. Everything else in study design is an attempt to construct a usable substitute for it.

A control group is that substitute. It is a set of cases that did not receive the thing, chosen so that they resemble the treated cases in every way that matters, so that the difference in outcomes can be attributed to the treatment rather than to the differences between the groups. Stated that way, the whole design question becomes visible: how do you get two groups that differ only in the one respect you care about?

What randomisation buys

Assigning cases to groups at random is the only method that balances the factors nobody thought of. Matching on known characteristics handles age, income and whatever else you measured, but it can do nothing about the variables that were never recorded and may never have been imagined. Randomisation does not measure those variables. It distributes them, on average, evenly between the groups, which is enough.

That protection is probabilistic rather than absolute, and it becomes weaker as the groups get smaller. In a small trial, chance can easily leave the groups meaningfully different, which is one reason study size matters beyond the obvious point about precision. This is also why good reports include a table comparing the groups at the outset: it is a check on whether the randomisation happened to produce balance in the factors that were measured.

And randomisation only protects what happens after assignment. If people drop out differentially — if those doing badly leave one group faster — the groups stop being comparable and the benefit is lost partway through.

Why comparing to no group at all fails

Studies that measure people before and after an intervention, without any comparison group, are common because they are cheap, and they are vulnerable to several things at once. Conditions often improve on their own. People enter treatment when things are at their worst, so the next measurement tends to be better regardless. Knowing you are being observed changes behaviour. Measurement instruments produce different results the second time they are used.

Each of these can produce an apparent effect where none exists, and they can operate together. A control group handles all of them simultaneously, without anyone needing to anticipate which is at work, because whatever they are, they apply to both groups. That generality is the reason the design is so valuable and so hard to replace with cleverness.

What a control group cannot do

It cannot tell you whether the result applies to people unlike those studied, which is a question about the sample rather than the design. It cannot tell you whether the effect persists after the study ends. It cannot rescue an outcome measure that does not capture what anyone cares about, and a trial that measures a convenient proxy will produce a clean answer to the wrong question.

There is also the matter of what the average conceals. A trial reports what happened to a group, and a group average is consistent with everyone improving slightly, with most people unaffected while a few improved enormously, or with genuine benefit to some and harm to others. Those are very different worlds for a person deciding what to do, and the summary figure does not distinguish between them unless the report goes looking.

Nor does a control group help if the comparison is unfair in some other way. Comparing an intervention against nothing at all, when the alternative in real life is a different intervention, answers a question nobody faces. And when participants know which group they are in, expectation can produce real changes in reported outcomes, which is what blinding is for. Blinding and control are separate protections against separate problems, and a study can have one without the other.

Reading for it

When you encounter a claim that something works, the first question is what it was compared with. If the answer is nothing, or the same people beforehand, or a group that self-selected, the study can describe what happened but cannot say what caused it.

This is not a demand that everything be a randomised trial, which is often impossible and sometimes unethical. Plenty of important knowledge rests on other designs, and those designs have their own careful methods for approximating a counterfactual. The demand is only that the question be asked, because a study without any answer to it is reporting a sequence of events rather than an effect.