Statistics From Zero · Lesson 11 of 22
Conditional Probability and Bayes, Explained With a Car Alarm
In one sentence: The chance of something depends on what you already know, and when the thing is rare, even a very accurate alarm is usually a false alarm.
The idea
Most probabilities you meet in real life come with a condition attached: the chance of rain given dark clouds, the chance a stock falls given bad news, the chance of a break-in given that the alarm is screaming. That "given" changes everything, and it's where most people, including plenty of experts, go wrong.
The classic mistake is to mix up two questions that sound the same: "how often does the alarm go off when there's a burglar?" and "how often is there a burglar when the alarm goes off?". They can have wildly different answers.
Uncle Gary's car alarm
Uncle Gary has fitted a new car alarm. The box says it detects 99% of break-in attempts. Gary is thrilled. Every time it goes off at night, he runs into the street in his dressing gown, holding a torch and a baseball bat he has never used for baseball.
He has done this 34 times this year. He has found: the wind, a cat, the wind again, Kevin, and on one memorable night a hedgehog. No burglars.
Gary concluded the alarm is broken. It isn't. It's working exactly as described. Gary just answered the wrong question.
Count people, not percentages
Percentages of percentages confuse everyone. Real counts don't. So imagine 10,000 nights on Gary's street, and suppose:
- A real break-in attempt happens on 1 night in 1,000. That's 10 nights.
- The alarm catches 99% of real attempts: it goes off on about 10 of those 10 nights.
- But it also goes off by mistake on 5% of quiet nights (wind, cats, hedgehogs). There are 9,990 quiet nights, so that's about 500 false alarms.
So the alarm goes off on about 510 nights, and only about 10 of those are real. When Gary hears it, the chance there's actually a burglar is about 10 out of 510, roughly 2%. The alarm is right 99% of the time when there's a burglar, and wrong 98% of the time when it rings.
Both statements are true. They answer different questions. And the reason they're so far apart is that real burglaries are rare. When the thing you're looking for is rare, the false alarms from the huge pile of ordinary nights swamp the real ones.
That's Bayes' theorem
What you just did has a famous name: Bayes' theorem. It's a rule for updating a belief when you get new evidence. You start with how common something is before any evidence (the base rate, or prior: 1 night in 1,000), you look at how well the evidence separates the cases (99% caught, 5% false alarms), and you end with the updated chance (about 2%).
The formula in your textbook does exactly the counting above, with letters. If the formula scares you, don't use it. Count 10,000 imaginary people or nights, split them, and read off the answer. Statisticians call this "natural frequencies", and it's how many of them check their own work.
Where this hides in real life
- Spam filters that flag your aunt's emails, because there are far more normal emails than spam.
- Security scanners at airports that beep at belt buckles all day.
- Trading signals: an indicator that "fired before 9 of the last 10 crashes" sounds amazing. But crashes are rare. If it also fires on 5% of ordinary days, almost every time it fires, nothing happens. Ask how often it fires when nothing follows. We tested a version of this with Grandpa's knee.
Try it yourself
Aunt Linda gets 1,000 emails a month. 10% of them are spam. Her filter catches 90% of spam, but it also wrongly flags 2% of normal emails.
Of the emails that end up in her spam folder, what share are really spam? And of the emails left in her inbox, what share are spam?
Show the answer
Count them. 100 are spam: the filter flags 90 and misses 10. 900 are normal: it wrongly flags 2%, which is 18.
The spam folder holds 90 + 18 = 108 emails, of which 90 are spam: about 83%. So roughly 1 in 6 emails in the spam folder is real mail, which is why Linda's cousin's wedding invitation was in there.
The inbox holds 10 spam + 882 normal = 892 emails, so only about 1.1% is spam. Spam here is much more common than burglars, so the filter does far better than Gary's alarm. Same kind of tool, very different base rate.
The words your lecturer uses
| Lecturer says | It means |
|---|---|
| Conditional probability, P(A | B) | The chance of A, given that you already know B happened. Read the bar as "given" |
| Bayes' theorem | The rule for turning P(B | A) into P(A | B) using the base rate |
| Prior (base rate) | How likely something is before you see the new evidence |
| Posterior | How likely it is after you've taken the evidence into account |
| Sensitivity (true positive rate) | How often the test says yes when the answer really is yes (the alarm's 99%) |
| False positive rate | How often it says yes when the answer is really no (the 5% of quiet nights) |
| Base rate fallacy | Ignoring how rare something is, and so trusting a positive result far too much |
Three things to remember
- "How often does the test say yes when it's true?" and "how often is it true when the test says yes?" are different questions.
- When the thing you're looking for is rare, most alarms are false alarms, even from a good test.
- If the formula confuses you, count 10,000 imaginary cases and split them. It's the same maths.
Part of Statistics From Zero, a free beginner course by WrongStat. Not a substitute for your lecturer, but hopefully a good translator.