Multiple testing
If you test many ideas, some will look good by chance alone. A multiple-testing correction raises the bar to account for all the tries.
In plain English
One test with a 1 in 20 chance of a fluke is fine. Run 72 tests and you should expect a few flukes. The correction asks each result to clear a higher bar, depending on how many things you tried.
Uncle Gary explains
Twenty people at the barbecue each try to throw a sausage into the bin from across the garden. One makes it. He is not a sausage-throwing genius. With twenty tries, somebody was always going to.
Why it matters for your money
We tested 72 versions of the moving-average cross. A few looked good at first. After correcting for the number of tries, none survived.
Where we tested it
- Does the Golden Cross Work? We Tested 72 Strategies
- Does Buying the Dip Work? We Checked Which Dips
- Does Volume Confirm the Trend? Sometimes It Contradicts It
- The Birthday Paradox Explained at a Wedding
- Lesson 8: Is the Coin Rigged? Hypothesis Testing and the P-Value
- Lesson 10: How to Lie With Statistics (So You Can Catch It)
- Is Fantasy Football Luck or Skill? Kevin's Three Titles, Tested
- Do Trading Strategies Work? Here Are All 430 Versions We Tested
- Lesson 21: What Is ANOVA? Comparing Three or More Averages at Once
Related terms
All glossary terms. Educational content, not financial advice.