The positive test
A disease affects 1% of people. A test for it is 95% accurate both ways. You test positive. How worried should you be? Picture a town of 10,000 people and count.
From the slides: Why should anyone care about chance? and Probability and what happens when you learn something.
P(sick | positive)
—
Positive tests in the town
—
P(healthy | negative)
—
What to notice
- With the default numbers only about 16% of positive tests belong to sick people. The test is fine; the disease is rare, and 5% of 9,900 healthy people (495 false alarms) swamps 95% of 100 sick people (95 true positives).
- Make the disease common (10% or 20%) and a positive test becomes far more convincing. The prior matters as much as the test.
- Push P(negative | healthy) to 99.5% and watch the false alarms collapse: for rare diseases, specificity is what counts.
- The tile at the top is Bayes' theorem: P(sick | +) = P(+ | sick)P(sick) / [P(+ | sick)P(sick) + P(+ | healthy)P(healthy)]. Counting people in a town is the same calculation, and easier.