Myth #6 · Track records
Do Trading Strategies Work? Here Are All 430 Versions We Tested.
Verdict: a long, impressive track record proves very little on its own. Across nine of our studies, 430 versions of popular trading ideas were tested, and not one beat a random stock bought on the same day once we corrected for how many versions we tried. Several of them looked brilliant before that correction. Any of those could have been sold as a system.
Uncle Gary has bought a course. The man in the video has a strategy that "has worked for 20 years". There's a chart. It goes up and to the right, steadily, like a well-behaved escalator.
Aunt Linda asked one question: "How many strategies did he try before this one?" The video did not say. The man in the video had a boat, which Linda feels is also an answer.
We can't see his drawer of failed strategies. But we can show you ours.
Our graveyard
Every row below is a real study we ran on daily data for US stocks, each with costs and a fair comparison: the same rules applied to a random stock on the same day, or the same stocks without the signal. "Survived" means it still beat that comparison after correcting for the number of versions tested.
| The idea | Versions tested | Survived |
|---|---|---|
| Moving-average crosses with stops | 72 | 0 |
| Breakouts after quiet, low-trend periods | 72 | 0 |
| A "billionaire's method": time to recover, first failure, profit-taking panic | 108 | 0 |
| Five ways to measure trend strength, plus a vote | 60 | 0 |
| "Skip the trade when X" filters on top of existing rules | 28 | 0 |
| Buying pullbacks in strong stocks | 24 | 0 |
| Volume confirming the move | 24 | 0 |
| An oscillator at its lowest point of the period | 24 | 0 |
| Overnight versus daytime returns | 18 | 0 |
| Total | 430 | 0 |
Not in the table, because they weren't counted in versions: support levels and Fibonacci (seven families of chart levels, none beat a randomly shifted level), AI reading the news (599 stories, correlation 0.01) and Wikipedia page views as a signal of attention (491 stocks, correlation near zero). Same result.
Some of them looked brilliant
Here's the uncomfortable part: the graveyard isn't full of obvious duds. Plenty of tombstones had a great chart once.
- Among the 72 moving-average versions, one (a 20/50-day exponential cross with a 2-ATR stop) was profitable, stable in both halves of the data and statistically significant on its own, with a t-statistic of 2.55. Sold alone, it's a course. Counted as one of 72, its corrected p-value is 0.77. The bar is 0.05.
- In the "billionaire's method", 32 of 108 comparisons crossed the usual significance line, when about 5 would by chance. Every one of them turned out to be a background effect unrelated to the idea itself, and none survived the correction or the costs.
- In the breakout study, 33 versions were "significant". All 33 pointed the wrong way: worse than doing nothing.
If we were selling courses, we'd have had a new one every month.
Why the winner is almost always luck
The usual bar for "probably not luck" is 5%: a result that pure chance would produce one time in twenty. That's fine for one test. It's a disaster for 430.
Test 430 versions that do nothing at all, and about 21 will clear the 5% bar anyway. Pick the best of those, draw its chart, and you have a strategy that "has worked for 20 years". It's the same arithmetic as Kevin's fantasy league titles: with enough tries, somebody always has a spectacular record.
That's why we correct for the number of tries. The simplest correction (named after a mathematician, Bonferroni) divides the 5% bar by the number of versions. It's strict, maybe too strict. But a real edge should survive it, and the ones in the table didn't.
430 tested versions, counted honestly
Statistics for noobs: the drawer you don't see
A track record is only as good as the number of failures it was chosen from. If someone tested one idea and it worked for 20 years, that's interesting. If they tested 500 and are showing you the best, it's a lottery ticket that already won, and it won't win again just because you bought it.
Four questions to ask about any strategy: How many versions were tested? Was it checked on data it wasn't built on? Does it beat a random stock on the same days? Does it survive costs? If the seller can't answer, the chart is decoration.
The moral
Gary watched the whole course. Then he asked the man in the video, in the comments, how many strategies he had tested. The comment was deleted. Gary considers this the most informative part of the course.
We'll keep adding to the graveyard. Every test on this site is compared with a random stock and with the S&P 500, the bar none of these 430 versions cleared. That's not advice to buy it. It's just the bar.
What this test can't tell you
- Our tests, our versions. Each idea was turned into exact rules by us. A different version, another market or intraday data might behave differently.
- Daily data, US stocks. Everything here uses daily prices of US stocks listed today, which brings survivorship bias: companies that disappeared are missing.
- The correction is strict. Bonferroni can hide a weak real effect. That's why we also look at whether results hold in both halves of the data and whether they point the right way, and those checks failed too.
- Absence of evidence. "Nothing survived our test" isn't proof that no strategy can ever work. It's proof that these ones didn't show it.
Simulations on historical data, not real trades. Daily prices of US stocks listed today, mostly 2010 to 2026, with trading costs. Historical results do not guarantee future returns. This is educational content, not financial advice.