Simpson’s paradox, the Berkeley admissions row, and how the overall number can tell the opposite story
Let’s start with two colleagues.
Priya and Tom both handle enquiries. The enquiries come in two kinds: easy ones, which mostly get resolved, and hard ones, which mostly don’t.
On the easy ones, Tom has the better record: he resolves 9 out of every 10, against Priya’s 8 out of 10. On the hard ones, Tom is also ahead: he cracks 25 in every 100, where Priya manages just 1 in 10. So Tom is better at the easy enquiries and better at the hard ones.
Then the quarterly figures come out, and Priya has resolved 74% of her enquiries overall. Tom has resolved 31%.
Tom is better at every single type of enquiry, and his overall score is less than half of Priya’s. He has not done anything wrong. He has simply been handed the hard pile, while Priya spent most of her quarter on the easy one. The headline number isn’t measuring who is better at the job. It’s measuring who got the nicer workload.
This is Simpson’s paradox: a trend that holds in every group can reverse, or vanish, when you add the groups together.
What happened at Berkeley
In 1973, the University of California, Berkeley looked at its graduate admissions and found an uncomfortable pattern. Around 44% of men who applied were admitted, against around 35% of women. On the face of it, that is a clear bias against women, and the kind of gap that quite reasonably starts an investigation.
So the university investigated, department by department. And the pattern fell apart. Within individual departments, there was no general bias against women. If anything, several departments admitted women at a slightly higher rate than men.
Both things were true at once. Women were admitted at a lower rate overall, and women were not being treated worse within departments. The explanation was where people had applied. Women had applied in greater numbers to the most competitive departments, the ones with low admission rates for everybody. Men had applied in greater numbers to departments that admitted most of their applicants. The overall gap wasn’t really about gender at all. It was about which queues people had joined.
The aggregate number pointed at discrimination. The detail pointed at the choice of subject. Acting on the headline alone would have meant trying to fix a problem in the wrong place entirely.
Why this happens
Simpson’s paradox turns up whenever a hidden variable sits underneath the groups and the groups are different sizes. In the enquiries example, the hidden variable is the difficulty of the workload. At Berkeley, it was the competitiveness of the chosen department. The totals quietly fold that hidden variable into the result, and the result is technically correct and genuinely misleading at the same time.
It is not a rare mathematical curiosity. It is a routine feature of any number that has been summed up from smaller groups, which is to say almost every number on almost every dashboard. Attainment gaps, satisfaction scores, retention rates, conversion rates, success rates by provider: all of them are totals sitting on top of subgroups that may be telling a different story, or the opposite one.
What to ask instead
When you are handed an aggregate figure, especially one that suggests someone is doing well or badly, a few questions help:
Has this been broken down, and by what? A total on its own can’t tell you whether the pattern holds underneath it.
Could the groups differ in a way that explains the gap? Different intakes, different courses, different entry routes, different starting points. The thing driving the headline is often not the thing the headline names.
Does the pattern survive when you split it? If a gap appears in the total but disappears inside every subgroup, the total is describing the mix, not the thing you think you are measuring.
And the practical one: am I about to act on the headline, or on what is actually happening underneath it?
The point
Aggregating data is not a trick or a distortion. It is necessary, and most of the time it is fine. But the act of adding things up can hide the very thing you most need to see, and it can do so while every individual number stays honest.
So when a single figure tells you that something is clearly going right, or clearly going wrong, it is worth asking what it looks like one level down. Because yes, the number might be technically correct. But what does it actually mean?
Read more in our Yes, but what does it actually mean? series