Where to Put the Armour: Abraham Wald's Bombers and Gary's Dented Minibuses
The short version: if you only study the things that came back, the damage you see is the damage they could survive. During the Second World War, Abraham Wald, working with the Statistical Research Group at Columbia University, analysed damage on returning planes and argued that armour should go where returning planes had few holes, because planes hit there tended not to return. The popular retellings vary in detail, but the lesson holds: the missing cases often matter most.
Uncle Gary has been put in charge of the village's three community minibuses. He walked round them in the depot and made a careful map of every dent: lots on the rear bumpers, plenty along the sides, a few on the roof from the car park barrier. Almost none on the front corners.
His report to the parish council recommended steel guards on the rear bumpers and the sides, "where the damage is". The council approved it unanimously, partly because the report was laminated. The mechanic, a quiet man called Stan, asked whether anyone had wondered where the fourth minibus was.
The minibus that isn't in the depot
The village used to have four minibuses. The fourth was hit on the front corner by a delivery lorry two years ago and written off. It isn't in the depot. Its damage isn't on Gary's map.
That's the whole problem. Gary studied the minibuses that came back. A dent on a rear bumper is survivable: you get a dent and keep driving. A hit on the front corner, where the steering and the engine are, tends to end the minibus. So the depot is full of dented bumpers and pristine front corners, not because front corners are rarely hit, but because front-corner hits don't come home. This is survivorship bias.
Abraham Wald and the planes
The same reasoning is famous from the Second World War. The statistician Abraham Wald worked with the Statistical Research Group at Columbia University, a team of mathematicians helping the US military. One question was how to protect aircraft: where should extra armour go, given that armour is heavy and can't go everywhere?
Damage on returning planes was concentrated in some areas and sparse in others. The obvious answer was to armour where the holes were. Wald's insight was that the holes showed where a plane could be hit and still come back. The places with few holes, such as the engines, were the places where hits brought planes down. So that's where protection would do most good. Popular retellings differ on the details (which planes, which air force, what happened next), but the logic is sound, and Wald's written work on estimating vulnerability from the damage on survivors is real.
Dents on the minibuses in the depot, by area (illustrative)
Finding the missing cases
The lesson isn't "do the opposite of the data". It's "ask what data is missing, and why". Wald's trick worked because he could reason about the missing planes: if hits were spread roughly evenly, then areas with too few holes among the survivors were where the missing planes had been hit. The absence was the evidence.
How to spot survivorship bias
1. Ask what didn't make it into the sample. Written-off minibuses, planes that didn't return, businesses that closed, patients who dropped out.
2. Ask why they're missing. If the reason for missing is related to what you're studying, the survivors will mislead you.
3. Look for the gap. Where the survivors look suspiciously clean, the missing ones may tell the real story.
Where it costs you money
Books about the habits of successful companies study the companies that survived. The companies with the same habits that went bust aren't in the book. Fund performance tables often leave out funds that were closed. If you learn only from survivors, you learn which risks can be survived, not which ones are worth taking. That's why our tests include stocks that were later delisted.
The moral
Stan showed Gary the photographs of the fourth minibus. Gary studied them for a long time and revised his report: steel guards on the front corners, and a new rule that minibuses must not be parked next to the delivery depot. The council approved it unanimously, partly because it was laminated in a different colour. Gary has since started a map of the places where the minibuses are never dented. He calls it "the map of the missing", and he is very proud of it.
The minibuses and their dents are illustrative. The Wald story is summarised from historical accounts, which vary in detail. Educational content, not financial advice.