Your Application Has Been Rejected Because You Failed to Apply Successfully
The short version: '99% of complete applications approved' can be true while only about a third of all applications get approved, if the office itself decides which applications count as complete. In our illustrative example, 640 of 1,000 applications are ruled incomplete and never reach the statistic; 356 of the remaining 360 are approved. That's 98.9% of complete applications and 35.6% of all of them. Before trusting any success rate, ask what's in the denominator, and who decided.
Aunt Linda wanted to build a garden shed. The council's website said the permits office approves 99% of complete applications, so she filled in the form, attached the drawings and paid the fee.
Three weeks later a letter arrived. Her application was incomplete. It was missing Form C7, the Confirmation of Completeness.
Linda applied for Form C7. That request was also declined: Form C7 is only issued for complete applications, and the office had no complete application from her on file. She has the letters on the fridge, next to each other, where they explain each other forever.
How to approve 99% of applications
Here's an illustrative year at the office. The numbers are made up, but the arithmetic is the whole trick:
- 1,000 applications arrive.
- 640 are ruled incomplete at the front desk: a missing Form C7, a signature in the wrong colour, a drawing of a shed that looks, in the clerk's opinion, too much like a garage.
- The remaining 360 are declared complete. The office approves 356 of them.
356 out of 360 is 98.9%. That's the number on the website, and it's true. But 356 out of 1,000 is 35.6%. Same office, same year, same sheds. The only thing that changed is what you divide by.
The trick isn't in the counting. It's in the filter before the counting. Anything that doesn't fit the result is ruled out of the statistic before the statistic is calculated, so the statistic is never wrong. It's the same move as a "satisfied customers" survey sent only to customers who didn't ask for a refund.
Approval rate at an illustrative permits office
Who decides what counts
The second problem is who runs the filter. The permits office is judged on its approval rate, and the permits office decides which applications are complete. Every application it might reject can be called incomplete instead, which costs nothing and improves the rate. Nobody has to lie. The incentive does the work quietly. That's Goodhart's law: once a number becomes the target, people find the cheapest way to move the number.
The office's other statistic is just as impressive: an average decision time of 6 days. It's measured from the moment an application is declared complete. Linda's application has been waiting 11 weeks, but it hasn't started yet, so it doesn't count.
The applications you never see
The 640 incomplete applications don't appear anywhere. They aren't rejections, so they're not in the rejection rate. They aren't approvals, so they're not in the approval rate. Like the failed businesses missing from a list of success stories, they vanish before anyone counts. A success rate calculated only on survivors is survivorship bias with a rubber stamp.
How to read any success rate
1. Ask what's in the denominator. "99% of complete applications", "95% of customers who finished the course", "90% of patients who completed treatment". The words after "of" are where the result is decided.
2. Ask who decides what counts. If the people being measured also decide which cases are measured, expect the rate to look good.
3. Ask what happened to the rest. Find the number that dropped out before the counting started. Then compute the rate out of everyone who began.
Where it costs you money
A fund company that says "92% of our funds beat their benchmark" may be counting only the funds it still runs. The ones that did badly were closed or merged and left the denominator. A trading course that says "80% of graduates are profitable" may count only the students who finished it and answered the survey. Ask for the rate out of everyone who started, including the ones who quietly disappeared.
The moral
Linda built the shed anyway. A month later the council published its annual report: zero unauthorised garden structures were reported this year. The report form for an unauthorised structure asks for its permit number. Linda's shed doesn't have one, so it can't be reported, so it doesn't exist. It is, statistically, the most compliant shed in the county.
The permits office and all its numbers are fictional and illustrative; no real council is described. The arithmetic of choosing a denominator is real. Educational content, not financial advice.