The Discriminatory Power of Diagnostic Information from Discrete Medical Tests

A useful diagnostic test should provide information that helps to discriminate between competing hypotheses. But any practical diagnostic will be imperfect: both false positive and false negative indications are to be expected. So just how useful is a diagnostic test when it is, necessarily, imperfect? In [1], p. 44 shows a static, graphical example of how Bayes's theorem may be used to understand the factors determining the discriminatory power of diagnostic tests. This Demonstration is a dynamic version of that argument.
Let be the logical truth value (1 or 0) of a proposition about a state variable (e.g., a disease or health risk is present or absent), and let be the logical truth value (1 or 0) of a proposition about the outcome of an indicative imperfect diagnostic test (e.g., an X-ray or blood test measurement is either definitely positive or negative for this disease). From a statistical perspective there are three precise numerical inputs that feed into a coherent posterior inference about binary-valued after having observed the result of the binary-valued diagnostic signal : a sensitivity number, a specificity number, and a base rate number. The first two characterize uncertainty about the outcome of the diagnostic as a conditional probability under two different information conditions about the state . The sensitivity number expresses uncertainty about whether the diagnostic test will be positive, that is, , assuming that is true. The specificity number expresses an uncertainty about whether the diagnostic test for will be negative, that is, , assuming that is true. The third number, the base rate number, is a marginal or unconditional probability, , characterizing uncertainty about the binary state variable in the absence of, or prior to knowing, any diagnostic information .
The discriminatory power of diagnostic information can be measured by the levels and differences between two inverse conditional probability assessments, and , one for each possible diagnostic test result. This interactive Demonstration creates a graphical depiction of the inverse probabilities and as functions of the underlying sensitivity, specificity, and base rate inputs. A natural frequency representation of the full joint probability distribution over the random variables (, ) is provided in a truth table format above the graph, where the column entries are frequency counts or "cases" in a hypothetical population of a fixed size.


  • [Snapshot]
  • [Snapshot]
  • [Snapshot]


The truth table in the upper-right shows the four logical possibilities for the propositions and being true or false together. Frequencies or counts are specified implicitly by parameter choices for each of the four logical possibilities in the columns, with a total population count initially set at 100. There are a total of 20 cases in the first two columns, where the proposition is true (), the remaining 80 cases being associated with being false (). This gives a base rate for of 20 out of 100, or 20%, shown as a vertical line on the axis at 0.2 in the figure. Continuing to focus on the first two columns of the table, the sensitivity of the diagnostic test, , is set initially at 16 out of 20 or 80%. Shifting focus to the last two columns of the table, where , the specificity of the test, , is set initially at 56 out of 80, or 70%. From Bayes's theorem, curves showing the two inverse posterior probabilities for being true, one based on a positive diagnostic, , the other based on a negative diagnostic, , are plotted in the graph for all possible base rates. Both of these curves shift as the sensitivity and specificity parameters are changed, albeit quite asymmetrically. Changes in sensitivity have a dramatic impact on the inferences to be made from negative diagnostic signals, that is, on the dashed lower curve, , while changes in specificity have a dramatic impact on the inferences to be made from positive diagnostic signals, that is, on the solid upper curve, . The actual inverse probabilities for the initial selection of parameters are indicated by the labeled squares on the relevant curves—a 40% chance of with a positive diagnostic , and a 7% chance of with a negative diagnostic, . The upper set of sliders can be varied to illustrate the impact of changes in the underlying parameters, while the lower set of sliders can be used to create a benchmark set of results for comparison proposes—the benchmark being illustrated in the background as curves and points of lighter shading.
A richer explanation of Bayes's theorem in the 2×2 case is available in the Demonstration Bayes's Theorem and Inverse Probability. A comprehensive explanation of the natural frequency method of representing probability distributions in the 2×2 discrete case is available in the Demonstration Comparing Ambiguous Inferences when Probabilities Are Imprecise.
Snapshot 1: a reduced base rate (20% down to 5%)
Snapshot 2: an improved specificity (70% up to 95%)
Snapshot 3: an improved sensitivity (80% up to 95%)
[1] J. M. Bernardo and A. F. M. Smith, Bayesian Theory, New York: John Wiley & Sons, 1994.
    • Share:

Embed Interactive Demonstration New!

Just copy and paste this snippet of JavaScript code into your website or blog to put the live Demonstration on your site. More details »

Files require Wolfram CDF Player or Mathematica.