Bayes' theorem

See why a 99% accurate test can still be wrong most of the time.

Has it, tests + (4)
Has it, tests − (0)
Doesn’t, tests + (20)
Doesn’t, tests − (376)

Bayes’ theorem updates a belief with evidence. The surprise: when a condition is rare, most positive results come from the huge group of people who don’t have it.

Counting people (natural frequencies) makes it intuitive: compare the red dots with the orange ones.

P(A | +) = P(+ | A) · P(A) / P(+) P(+) = P(+ | A)·P(A) + P(+ | not A)·P(not A)

Scenario

Base rate (prevalence)1%

Sensitivity: P(+ | has it)99%

Specificity: P(− | doesn’t)95%

Positive tests: 24
Truly positive: 4
P(has it | +): 16.7%

Of everyone who tests positive, 4 of 24 really have it — about 17% in this crowd of 400.

Bayes’ theorem updates a belief with evidence. The surprise: when a condition is rare, most positive results come from the huge group of people who don’t have it.

Counting people (natural frequencies) makes it intuitive: compare the red dots with the orange ones.

P(A | +) = P(+ | A) · P(A) / P(+) P(+) = P(+ | A)·P(A) + P(+ | not A)·P(not A)

Things to try

With the rare disease preset, how likely is a positive patient to actually be ill?
Push sensitivity to 100%. Does the posterior improve much? Now raise specificity instead.
Raise the base rate to 30%. Why does the same test suddenly become trustworthy?
Find settings where P(has it | +) is exactly about 50%.

Test yourself: 5-minute drill Next: PageRank