The story of the positive test: probability, base rates and why surprising results are often wrong

Mira runs a small clinic in a mountain town. One afternoon a farmer, Pema, arrives pale and anxious. A screening test for a rare condition has come back positive. “The leaflet says the test is 90 percent accurate,” Pema says. “So I almost certainly have it. Don’t I?”

Mira has seen this moment before. She takes a pencil and a sheet of paper, and instead of using formulas, she draws a crowd.

Probability, in one sentence

Probability is a number between 0 and 1 (or 0 and 100 percent) that says how likely something is. A fair coin landing heads has a probability of 0.5. A day with no rain in the dry season might be 0.9. When something is certain, the probability is 1. When it is impossible, it is 0.

A powerful trick, which Mira uses every day, is to turn probabilities into counts of people. “One in a hundred” is easier to picture than “0.01”, and counts are much harder to misunderstand.

Mira draws a crowd

The condition affects about 1 percent of people in the area. That is the base rate: how common it is before anyone is tested. Mira imagines 1,000 people from the town.

• Of the 1,000, about 10 truly have the condition, and about 990 do not.
• The test catches 90 percent of true cases. So of the 10 who have it, 9 test positive, and 1 is missed.
• The test also gives a false alarm for about 5 percent of healthy people. Five percent of 990 is roughly 50. So about 50 healthy people test positive.

Now count the positives. There are 9 true ones and 50 false ones, so 59 positive results in total. Pema is one of those 59. How many of them truly have the condition? Nine out of 59, which is about 15 percent.

The “90 percent accurate” test, when it says positive, is right only about one time in seven. Pema probably does not have the condition. A second, more careful test is needed, and Mira explains this calmly. Pema’s face softens.

Why it happens

When something is rare, there are very few people who can produce a true positive, and a great many who can produce a false one. Even a small false alarm rate, applied to a large healthy crowd, creates a pile of false positives that swamps the few real ones. The rarer the event, the worse this gets.

Change the story, and see the effect. If the condition affected 30 percent of people, the same test would be right about 87 percent of the time when it said positive. Same test, different crowd, very different meaning.

The idea behind Bayes’ rule

What Mira did on paper is the heart of Bayes’ rule, a way of updating beliefs with evidence. You start with what you believed before, the base rate. You see new evidence, the test result. You combine the two to get what you should believe now. Written without symbols: the chance you really have it, given a positive test, depends on how common it is, how good the test is at catching it and how often it cries wolf.

• Prior: how likely was it before the test? Here, 1 in 100.
• Evidence: how much more likely is a positive result if you have it than if you do not? Here, 90 percent versus 5 percent, so about 18 times.
• Posterior: how likely is it now? Here, about 15 in 100.

Where this shows up in AI, maps and 3D

• Spam and fraud filters: if only 1 in 1,000 transactions is fraud, a detector that flags 1 percent of honest transactions will raise many more false alarms than real catches.
• Satellite detection: a model that finds illegal mining, deforestation or buildings has the same problem when the thing it looks for is rare in the landscape. Evaluate with precision and recall, not accuracy alone.
• Medical imaging and screening programmes: positive results need confirmation, because most positives in a healthy population are false.
• AI assistants: a model’s “confidence” is not the same as the probability that it is right, and it ignores the base rate of questions it handles well.
• Weather: “30 percent chance of rain” is a probability about repeated similar days, not a statement that it will rain on 30 percent of the sky.
• Tracking in AR and VR: systems combine noisy sensor readings with what they expect, a close cousin of Bayes’ rule, to estimate where you are.

The formula behind the story

Mira’s crowd drawing is Bayes’ rule done with people instead of symbols. Written in symbols, it looks like this.

In the story: the top is 0.90 × 0.01 = 0.009. The bottom adds the false alarms, 0.05 × 0.99 = 0.0495, to get 0.009 + 0.0495 = 0.0585. So P(D | +) = 0.009 ÷ 0.0585 ≈ 0.154, about 15%. It is the same answer as 9 true positives out of 59 positives in Mira’s crowd.

In the story: 1 to 99, multiplied by 18, is 18 to 99, which is about 15% as a probability (18 ÷ 117). The test multiplies your belief by 18, but a belief that started tiny stays modest.

Try it yourself

1. 01Pick a rare event, such as 1 in 200.
2. 02Imagine 10,000 people. How many truly have it?
3. 03Suppose a test catches 95 percent of them and falsely flags 2 percent of the rest. How many positives are true, and how many false?
4. 04What share of all positives is true? Surprised?

Frequently asked questions

Does this mean tests are useless?

No. They are valuable, especially for ruling things out and for guiding the next step. The lesson is to interpret a positive result in light of how common the condition is, and to confirm it when it matters.

What is the difference between sensitivity and specificity?

Sensitivity is the share of true cases the test catches. Specificity is the share of non-cases the test correctly clears. A good screening test needs both to be high.

Is a 5 percent false positive rate a lot?

It depends on the crowd. With a rare condition and a big healthy population, it produces many false alarms. That is why alerts are tuned and confirmed.

How can I check my own reasoning?

Turn every probability into counts of people, as Mira did. If the counts and your gut feeling disagree, trust the counts.

Mira’s pencil sketch did more than a lecture could. It replaced a frightening number with a clear picture, and a picture is often the best tool in probability. When you hear that something is “90 percent accurate”, remember the crowd, and ask how many people stand in it.