9846

The Discriminatory Power of Diagnostic Information from Discrete Medical Tests

A useful diagnostic test should provide information that helps to discriminate between competing hypotheses. But any practical diagnostic will be imperfect: both false positive and false negative indications are to be expected. So just how useful is a diagnostic test when it is, necessarily, imperfect? In [1], p. 44 shows a static, graphical example of how Bayes's theorem may be used to understand the factors determining the discriminatory power of diagnostic tests. This Demonstration is a dynamic version of that argument.
Let be the logical truth value (1 or 0) of a proposition about a state variable (e.g., a disease or health risk is present or absent), and let be the logical truth value (1 or 0) of a proposition about the outcome of an indicative imperfect diagnostic test (e.g., an X-ray or blood test measurement is either definitely positive or negative for this disease). From a statistical perspective there are three precise numerical inputs that feed into a coherent posterior inference about binary-valued after having observed the result of the binary-valued diagnostic signal : a sensitivity number, a specificity number, and a base rate number. The first two characterize uncertainty about the outcome of the diagnostic as a conditional probability under two different information conditions about the state . The sensitivity number expresses uncertainty about whether the diagnostic test will be positive, that is, , assuming that is true. The specificity number expresses an uncertainty about whether the diagnostic test for will be negative, that is, , assuming that is true. The third number, the base rate number, is a marginal or unconditional probability, , characterizing uncertainty about the binary state variable in the absence of, or prior to knowing, any diagnostic information .
The discriminatory power of diagnostic information can be measured by the levels and differences between two inverse conditional probability assessments, and , one for each possible diagnostic test result. This interactive Demonstration creates a graphical depiction of the inverse probabilities and as functions of the underlying sensitivity, specificity, and base rate inputs. A natural frequency representation of the full joint probability distribution over the random variables (, ) is provided in a truth table format above the graph, where the column entries are frequency counts or "cases" in a hypothetical population of a fixed size.

SNAPSHOTS

  • [Snapshot]
  • [Snapshot]
  • [Snapshot]

DETAILS

The truth table in the upper-right shows the four logical possibilities for the propositions and being true or false together. Frequencies or counts are specified implicitly by parameter choices for each of the four logical possibilities in the columns, with a total population count initially set at 100. There are a total of 20 cases in the first two columns, where the proposition is true (), the remaining 80 cases being associated with being false (). This gives a base rate for of 20 out of 100, or 20%, shown as a vertical line on the axis at 0.2 in the figure. Continuing to focus on the first two columns of the table, the sensitivity of the diagnostic test, , is set initially at 16 out of 20 or 80%. Shifting focus to the last two columns of the table, where , the specificity of the test, , is set initially at 56 out of 80, or 70%. From Bayes's theorem, curves showing the two inverse posterior probabilities for being true, one based on a positive diagnostic, , the other based on a negative diagnostic, , are plotted in the graph for all possible base rates. Both of these curves shift as the sensitivity and specificity parameters are changed, albeit quite asymmetrically. Changes in sensitivity have a dramatic impact on the inferences to be made from negative diagnostic signals, that is, on the dashed lower curve, , while changes in specificity have a dramatic impact on the inferences to be made from positive diagnostic signals, that is, on the solid upper curve, . The actual inverse probabilities for the initial selection of parameters are indicated by the labeled squares on the relevant curves—a 40% chance of with a positive diagnostic , and a 7% chance of with a negative diagnostic, . The upper set of sliders can be varied to illustrate the impact of changes in the underlying parameters, while the lower set of sliders can be used to create a benchmark set of results for comparison proposes—the benchmark being illustrated in the background as curves and points of lighter shading.
A richer explanation of Bayes's theorem in the 2×2 case is available in the Demonstration Bayes's Theorem and Inverse Probability. A comprehensive explanation of the natural frequency method of representing probability distributions in the 2×2 discrete case is available in the Demonstration Comparing Ambiguous Inferences when Probabilities Are Imprecise.
Snapshot 1: a reduced base rate (20% down to 5%)
Snapshot 2: an improved specificity (70% up to 95%)
Snapshot 3: an improved sensitivity (80% up to 95%)
[1] J. M. Bernardo and A. F. M. Smith, Bayesian Theory, New York: John Wiley & Sons, 1994.
    • Share:

Embed Interactive Demonstration New!

Just copy and paste this snippet of JavaScript code into your website or blog to put the live Demonstration on your site. More details »

Files require Wolfram CDF Player or Mathematica.









 
RELATED RESOURCES
Mathematica »
The #1 tool for creating Demonstrations
and anything technical.
Wolfram|Alpha »
Explore anything with the first
computational knowledge engine.
MathWorld »
The web's most extensive
mathematics resource.
Course Assistant Apps »
An app for every course—
right in the palm of your hand.
Wolfram Blog »
Read our views on math,
science, and technology.
Computable Document Format »
The format that makes Demonstrations
(and any information) easy to share and
interact with.
STEM Initiative »
Programs & resources for
educators, schools & students.
Computerbasedmath.org »
Join the initiative for modernizing
math education.
Step-by-step Solutions »
Walk through homework problems one step at a time, with hints to help along the way.
Wolfram Problem Generator »
Unlimited random practice problems and answers with built-in Step-by-step solutions. Practice online or make a printable study sheet.
Wolfram Language »
Knowledge-based programming for everyone.
Powered by Wolfram Mathematica © 2014 Wolfram Demonstrations Project & Contributors  |  Terms of Use  |  Privacy Policy  |  RSS Give us your feedback
Note: To run this Demonstration you need Mathematica 7+ or the free Mathematica Player 7EX
Download or upgrade to Mathematica Player 7EX
I already have Mathematica Player or Mathematica 7+