Probability of Being Sick After Having Tested Positive for a Disease (Bayes's Rule)
For rare diseases, people tend to intuitively overestimate the probability of being sick after having received a positive test result. This probability is calculated using Bayes's theorem from some of the conditional probabilities involved in the scenario. Understanding these probabilities, such as the probability of being sick when having received a positive test result, expressed as P(sick| positive test), can become easier using a contingency table like the one shown. Given properties of the test, the probability of the disease, and the size of the reference group, the table shows the number of people falling into the four different possible categories. The conditional probability of being sick after having received a positive test result is simply the ratio of the people that are sick and that have tested positive to the total number of people having tested positive.
Even very good medical tests for diseases have—usually very small—probabilities for false positive and false negative results. For rare diseases, the number of healthy people getting a false positive test (light yellow background) can become much larger than the number of sick people getting a positive test (pink background).
Recommended reading on the subject:
G. Gigerenzer, Reckoning with Risk: Learning to Live with Uncertainty, London, UK: Penguin, 2002.