Suppose you take a hypothetical screening test for a rare disease. The disease prevalence is 1%. The test catches 90% of people who have it, giving it 90% sensitivity. It also returns a positive result for 5% of healthy people, giving it 95% specificity and a 5% false-positive rate.

Your result is positive. It is tempting to grab the 90% figure and conclude that you are 90% likely to be sick. But 90% answers, “If someone has the disease, how likely are they to test positive?” You need the reverse: “If someone tests positive, how likely are they to have the disease?” Reversing the direction can radically change the answer.

This is a common form of the base rate fallacy: vivid case-specific evidence pushes aside the event's original frequency, or base rate.

These measures are not interchangeable. Sensitivity is P(positive | disease). Specificity is P(negative | no disease). The false-positive rate is P(positive | no disease), which equals 1 minus specificity. Positive predictive value is the quantity you actually want: P(disease | positive). Overall accuracy is the proportion of all classifications that are correct, and it changes with the mix of diseased and healthy people in the sample. Hearing that a test is “99% accurate” without knowing the sample composition or confusion matrix is generally not enough to calculate the probability of disease after a positive result.