Statistics From Zero · Lesson 17 of 22
The Central Limit Theorem: Why Averages Become Bell Curves
In one sentence: Average enough independent random things together and the average follows a bell curve, whatever shape the individual things had, and it gets narrower in a predictable way as you add more.
The idea
In Lesson 5 you met the bell curve, and you were told it turns up everywhere. This lesson explains why. The reason has a grand name, the central limit theorem, and a simple message: averages behave nicely even when individual things don't.
Take lots of independent random things, add them up or average them, and the result follows a bell curve. It doesn't matter much whether the individual things were flat, lopsided or weird. Averaging smooths them into a bell.
Board-game night
Uncle Gary has bought a board game in which, at one point, you roll thirty dice at once and use the average. Gary thinks this is ridiculous. "A die can land on anything from 1 to 6," he says. "So the average of thirty dice can be anything from 1 to 6 too. Pure chaos."
Aunt Linda rolled the thirty dice twenty times. The average was always somewhere between about 3 and 4. Gary accused the dice of being boring.
From flat to triangle to bell
- One die: every face from 1 to 6 is equally likely. Flat. No bell anywhere.
- Two dice added: there's only one way to get 2 (1 and 1) but six ways to get 7. Totals near the middle are much more likely than totals at the edges. The shape is a triangle peaking at 7.
- Thirty dice averaged: getting an average near 1 needs almost every die to land on 1, which essentially never happens. Most combinations land near the middle. The shape is now a clear bell curve centred on 3.5.
The dice didn't change. Only the number of them averaged. That's the central limit theorem in one picture: the flat shape of one die disappears, and the bell of the average appears.
How narrow does it get?
The second half of the theorem is just as useful. The spread of the average shrinks in a predictable way: it's the spread of one item divided by the square root of how many you average.
- One die has a standard deviation of about 1.71.
- The average of 30 dice has a standard deviation of 1.71 ÷ √30 = about 0.31.
- So about 95% of the time, the average of 30 dice lands between 3.5 - 2 × 0.31 and 3.5 + 2 × 0.31: roughly 2.9 to 4.1. We simulated 100,000 rolls of thirty dice and 95.1% landed in that range.
Linda's results weren't boring. They were the theorem.
That spread of an average is exactly the standard error from Lesson 6. And the square root is why polls need four times as many people to halve their margin of error: √4 = 2.
Why this matters so much
Almost everything in Lessons 6 to 16 quietly relies on this theorem. Confidence intervals (Lesson 9) use "plus or minus two standard errors" because the average follows a bell curve. The t-test (Lesson 14) works on averages for the same reason. That's why you can test averages of things that are nothing like a bell curve themselves, such as incomes, waiting times or trade results.
Where it fails
- Small samples of very lopsided things. Averages of 5 incomes, where one might be a billionaire's (Lesson 2), won't look like a bell. You need more data the more lopsided the original is.
- Things that aren't independent. The theorem assumes each item is a separate roll. Stock returns on crash days aren't: everything falls together. That's one reason markets produce more extreme days than a bell curve predicts (the "fat tails" of Lesson 5).
- It's about the average, not the individuals. The average of thirty dice is bell-shaped; a single die is still flat. A bell curve of averages tells you nothing about any one roll.
Try it yourself
Cousin Kevin's lemonade customers buy very different amounts: most buy one cup, some buy five for their family. The number of cups per customer has a standard deviation of about 1.5 and is very lopsided.
1. What's the standard deviation of the average cups per customer over a day with 100 customers? 2. Can Kevin use a bell curve to plan around that daily average, even though single customers are lopsided?
Show the answer
1. 1.5 ÷ √100 = 1.5 ÷ 10 = 0.15 cups.
2. Roughly yes. With 100 customers, the average is close to a bell curve even though individual customers aren't. As long as customers decide independently. If a coach party of 40 arrives together, they're not 40 independent customers, and the bell curve stops being a good guide that day.
The words your lecturer uses
| Lecturer says | It means |
|---|---|
| Central limit theorem (CLT) | The average of many independent random values follows a bell curve, whatever their own shape |
| Sampling distribution | The shape you'd get if you repeated the sampling many times and plotted all the averages |
| Standard error | The standard deviation of that sampling distribution: σ ÷ √n |
| Independent and identically distributed (i.i.d.) | Each value is a separate draw from the same kind of randomness; the theorem's main assumption |
| Skewed (lopsided) distribution | One with a long tail on one side; needs larger samples before the average looks like a bell |
| Rule of thumb n ≥ 30 | A common, rough guide for when the average is "bell-shaped enough"; very lopsided data needs more |
Three things to remember
- Averages of many independent things follow a bell curve, even when the things themselves don't.
- The spread of an average shrinks with the square root of the sample: four times the data, half the spread.
- It needs independence. When everything moves together, as in a crash, the bell curve stops protecting you.
Part of Statistics From Zero, a free beginner course by WrongStat. Not a substitute for your lecturer, but hopefully a good translator.