Are Online Reviews Trustworthy? Are Online Reviews Trustworthy? Five Stars From Seven Reviews
The short version: a bar with 4.9 stars from 7 reviews tells you much less than one with 4.6 from 2,300. Small samples swing wildly, and the first few reviews are often written by people who love the owner.
Kevin found the perfect place for Friday night: a new cocktail bar with a flawless 5.0 rating. The old pub round the corner only has 4.5.
The cocktails were warm. The DJ was the owner's nephew, who only knew one song. Afterwards Kevin checked the reviews. There were three. One was from the owner's mum, one from the owner, and one from a man called Gary, who claims he's never been there.
The old pub's 4.5 came from 1,800 reviews.
Why the number matters more than the stars
An average from a handful of reviews swings wildly with every new one. Watch what a single unhappy customer does:
| Place | Rating now | After one 1-star review |
|---|---|---|
| New bar, 7 reviews | 4.9 | 4.4 |
| Old pub, 2,300 reviews | 4.6 | 4.6 |
The big number barely moves, because it's built on a big sample. The small one is mostly noise, plus the owner's mum.
There's a second problem: who writes reviews at all. Mostly people who had a wonderful night or a terrible one. The quiet majority who had a perfectly fine evening go home and say nothing. So even a big pile of reviews is a slightly odd sample. A small pile is an odd sample that also happens to be tiny.
Rating before and after one 1-star review
A rough rule for real life
Look at the number of reviews before you look at the stars. With only a few dozen, treat the rating as a rumour. With hundreds or thousands, it starts to mean something.
Where it costs you money
A trader who won 12 trades in a row, a fund with a brilliant two-year record, a strategy that "never failed" in last month's backtest: they are all the new bar with 5.0 stars from three reviews.
That's why our tests use thousands of trades, and why a short winning streak should make you curious, not convinced.
The moral
Stars without a count are like a stock tip without a track record: shiny, confident and based on almost nothing. Kevin now goes to the old pub. The cocktails there are also warm, but at least 1,800 people agree they're fine.
Ratings in the table are rounded to one decimal place. Educational content, not financial advice.