goodenomics™

The evidence

Base-rate neglect: what a positive test really means

A check flags someone. The check is right most of the time. How likely is it that the person it flagged is guilty? It depends on something most people leave out: how rare the thing was to begin with.

A scene from The Review, the journey where the game asks this question.
A scene from the game, as it is drawn: one handwritten line at a time.

Base-rate neglect

Ignoring how rare something is to begin with when judging what a positive test result means.

What the study found

In 1978, 60 students and doctors at Harvard Medical School were asked how likely a positive result is to mean disease, when the disease affects 1 in 1,000 people and the test has a 5% false-positive rate. Only 11 (18%) gave the right answer, about 2%. The most common answer, given by 27 (45%), was 95%.

Source: Casscells, W., Schoenberger, A., & Graboys, T. B. (1978). Interpretation by physicians of clinical laboratory results. New England Journal of Medicine 299(18), 999-1001. doi.org/10.1056/NEJM197811022991808

How to read this

These are the answers of particular people, asked in a particular way, at a particular time. A share of people is not a verdict: it tells you what they answered, not what is right.

Where the game asks it

The Review (What would change your mind?). The game’s versions with family, age, consent and combined pressures are new comparisons written for it. They are not copies of the experiments, and no population percentages are claimed for them.

Play The Review Free. No account. Your place is saved on your device.

More evidence