Wolfram Demonstrations Project

A useful diagnostic test should provide information that helps to discriminate between competing hypotheses. But any practical diagnostic will be imperfect: both false positive and false negative indications are to be expected. So just how useful is a diagnostic test when it is, necessarily, imperfect? In [1], p. 44 shows a static, graphical example of how Bayes's theorem may be used to understand the factors determining the discriminatory power of diagnostic tests. This Demonstration is a dynamic version of that argument.

Let

be the logical truth value (1 or 0) of a proposition about a state variable (e.g., a disease or health risk is present or absent), and let

be the logical truth value (1 or 0) of a proposition about the outcome of an indicative imperfect diagnostic test (e.g., an X-ray or blood test measurement is either definitely positive or negative for this disease). From a statistical perspective there are three precise numerical inputs that feed into a coherent posterior inference about binary-valued

after having observed the result of the binary-valued diagnostic signal

: a sensitivity number, a specificity number, and a base rate number. The first two characterize uncertainty about the outcome of the diagnostic

as a conditional probability under two different information conditions about the state

. The sensitivity number

expresses uncertainty about whether the diagnostic test

will be positive, that is,

, assuming that

is true. The specificity number

expresses an uncertainty about whether the diagnostic test

for

will be negative, that is,

, assuming that

is true. The third number, the base rate number, is a marginal or unconditional probability,

, characterizing uncertainty about the binary state variable

in the absence of, or prior to knowing, any diagnostic information

The discriminatory power of diagnostic information can be measured by the levels and differences between two inverse conditional probability assessments,

and

, one for each possible diagnostic test result. This interactive Demonstration creates a graphical depiction of the inverse probabilities

and

as functions of the underlying sensitivity, specificity, and base rate inputs. A natural frequency representation of the full joint probability distribution over the random variables (

) is provided in a truth table format above the graph, where the column entries are frequency counts or "cases" in a hypothetical population of a fixed size.

Contributed by: John Fountain and Philip Gunby

Slider Zoom
Automatic Animation

The truth table in the upper-right shows the four logical possibilities for the propositions

and

being true or false together. Frequencies or counts are specified implicitly by parameter choices for each of the four logical possibilities in the columns, with a total population count initially set at 100. There are a total of 20 cases in the first two columns, where the proposition

is true (

), the remaining 80 cases being associated with

being false (

). This gives a base rate for

of 20 out of 100, or 20%, shown as a vertical line on the

axis at 0.2 in the figure. Continuing to focus on the first two columns of the table, the sensitivity of the diagnostic test,

, is set initially at 16 out of 20 or 80%. Shifting focus to the last two columns of the table, where

, the specificity of the test,

, is set initially at 56 out of 80, or 70%. From Bayes's theorem, curves showing the two inverse posterior probabilities for

being true, one based on a positive diagnostic,

, the other based on a negative diagnostic,

, are plotted in the graph for all possible base rates. Both of these curves shift as the sensitivity and specificity parameters are changed, albeit quite asymmetrically. Changes in sensitivity have a dramatic impact on the inferences to be made from negative diagnostic signals, that is, on the dashed lower curve,

, while changes in specificity have a dramatic impact on the inferences to be made from positive diagnostic signals, that is, on the solid upper curve,

. The actual inverse probabilities for the initial selection of parameters are indicated by the labeled squares on the relevant curves—a 40% chance of

with a positive diagnostic

, and a 7% chance of

with a negative diagnostic,

. The upper set of sliders can be varied to illustrate the impact of changes in the underlying parameters, while the lower set of sliders can be used to create a benchmark set of results for comparison proposes—the benchmark being illustrated in the background as curves and points of lighter shading.

A richer explanation of Bayes's theorem in the 2×2 case is available in the Demonstration Bayes's Theorem and Inverse Probability. A comprehensive explanation of the natural frequency method of representing probability distributions in the 2×2 discrete case is available in the Demonstration Comparing Ambiguous Inferences when Probabilities Are Imprecise.

Snapshot 1: a reduced base rate (20% down to 5%)

Snapshot 2: an improved specificity (70% up to 95%)

Snapshot 3: an improved sensitivity (80% up to 95%)

[1] J. M. Bernardo and A. F. M. Smith, Bayesian Theory, New York: John Wiley & Sons, 1994.