9846

How Receiver Operating Characteristic Curves Work

Visually the ROC curve, shown in the top-right corner, is the shaded area under the right curve versus the shaded area under the left curve as the threshold parameter varies. A more detailed explanation now follows.
Let be a possible medical diagnostic for disease. For example, , could be eye pressure and the disease could be glaucoma. We suppose that the distribution of in healthy people is and in the diseased population it is , where . These curves are shown on the left. The receiver operating characteristic (ROC) curve can be used to visualize and quantify how useful is in the detection of this disease. We suppose that people are diagnosed healthy or diseased according as or . In the above diagram, we show the case where and . The ROC curve plots sensitivity versus specificity, where
,
.
Keeping fixed, as we vary the threshold parameter, , we trace out the ROC curve, shown in the upper-right corner. For any fixed value of , the point shown on the ROC curve corresponds to the two shaded areas.
The usefulness of the test depends on . The larger is, the larger the difference between the normal and diseased populations and the easier it is to detect disease. So the diagnostic test improves if increases. The or area under the ROC curve quantifies the usefulness of the test, . Increasing increases the . For large enough , . In our Demonstration, when . In this case, the test is useless and is equivalent to simply random guessing. Obviously, when , the test, , is worse than useless!
Sometimes the ROC curve is defined as the plot of FPR versus TPR, where FPR, the false positive rate, is defined as and TPR is the true positive rate, . Click the FPR checkbox to select this type of plot. In this plot, is the area to the right of for the healthy population and is shown as the colored area under the left curve. When this area overlaps with the curve of the diseased population on the right, the blended color is shown. Similarly TPR is the area to the right of in the diseased population and is shown as the colored area under the right curve; once again the overlapping area is shown as the blended color.

SNAPSHOTS

  • [Snapshot]
  • [Snapshot]
  • [Snapshot]
  • [Snapshot]
  • [Snapshot]

DETAILS

Denote the cumulative distribution functions of in the healthy and diseased populations by and . Then the tail functions are respectively , . It may be shown that , and
.
An explicit closed-form formula for AUC in the case of binormal populations is given in [1], p. 34.
The AUC can be interpreted as the probability that a randomly chosen member from the diseased population will have a higher than a randomly chosen member from the healthy population.
An interesting example of the use of ROC curves to compare classifiers is given in [2], figure 9.6, p. 316. See also the Demonstration "Uncertainty of Measurement and Diagnostic Accuracy Measures" for an illustration of the use of ROC to compare two diagnostic tests. In practice, ROC curves are often used to compare a number of diagnostic tests or classifiers.
The glaucoma example discussed in this Demonstration is presented in [4] and there are many similar examples in the references below. Two in-depth treatments of ROC curves are provided in [1] and [3].
[1] W. J. Krzanowski and D. J. Hand, ROC Curves for Continuous Data, CRC/Chapman & Hall, 2009.
[2] T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd ed., New York: Springer, 2009.
[3] M.S. Pepe, The Statistical Evaluation of Medical Tests for Classification and Prediction, New York: Oxford, 2003.
[4] J. A. Swets, R. M. Dawes, and J. Monahan, "Better Decisions through Science," Scientific American, 283, 2000 pp. 82–87.
    • Share:

Embed Interactive Demonstration New!

Just copy and paste this snippet of JavaScript code into your website or blog to put the live Demonstration on your site. More details »

Files require Wolfram CDF Player or Mathematica.









 
RELATED RESOURCES
Mathematica »
The #1 tool for creating Demonstrations
and anything technical.
Wolfram|Alpha »
Explore anything with the first
computational knowledge engine.
MathWorld »
The web's most extensive
mathematics resource.
Course Assistant Apps »
An app for every course—
right in the palm of your hand.
Wolfram Blog »
Read our views on math,
science, and technology.
Computable Document Format »
The format that makes Demonstrations
(and any information) easy to share and
interact with.
STEM Initiative »
Programs & resources for
educators, schools & students.
Computerbasedmath.org »
Join the initiative for modernizing
math education.
Step-by-step Solutions »
Walk through homework problems one step at a time, with hints to help along the way.
Wolfram Problem Generator »
Unlimited random practice problems and answers with built-in Step-by-step solutions. Practice online or make a printable study sheet.
Wolfram Language »
Knowledge-based programming for everyone.
Powered by Wolfram Mathematica © 2014 Wolfram Demonstrations Project & Contributors  |  Terms of Use  |  Privacy Policy  |  RSS Give us your feedback
Note: To run this Demonstration you need Mathematica 7+ or the free Mathematica Player 7EX
Download or upgrade to Mathematica Player 7EX
I already have Mathematica Player or Mathematica 7+