Uncertainty of Measurement and Areas Over and Under the ROC Curves

This Demonstration compares the ratios of the areas under the curve (AUC) and the ratios of the areas over the curve (AOC) of the receiver operating characteristic (ROC) plots of two diagnostic tests (ratio of the AUC of the first test to the AUC of the second test: blue plot, ratio of the AOC of the first test to the AOC of the second test: orange plot). The two tests measure the same measurand, for normally distributed healthy and diseased populations, for various values of the mean and standard deviation of the populations, and of the uncertainty of measurement of the tests. A normal distribution of the uncertainty is assumed. The uncertainty of the first test is defined. It is assumed that the uncertainty of the second test is greater than the uncertainty of the first test and varies up to a user defined upper bound. The six parameters that you can vary using the sliders are measured in arbitrary units.

The ROC plots are used in the evaluation of the clinical accuracy of a diagnostic test applied to a diseased and a non diseased population. They display the sensitivity of the test versus 1-specificity. Sensitivity is the fraction of the diseased population with a positive test, while specificity is the fraction of the non diseased population with a negative test. Therefore, the ROC plots display the true positive fraction versus the false positive fraction. Furthermore, the area under a ROC curve is used as an index of the diagnostic accuracy of the respective test. Consequently we can consider the area over the ROC curve as an index of its diagnostic inaccuracy. Actually the area over the ROC curve is equal to the definite integral from 0 to 1 of the false negative fraction versus the false positive fraction function.

This Demonstration could be useful in evaluating the maximum medically permissible uncertainty of measurement of a diagnostic test. For example, in the first snapshot the populations data describe a bimodal distribution of serum glucose measurements with a non diabetic and a diabetic population (Lim et al. 2002). The first test has a state of the art performance, while the second test has a greater uncertainty that varies. Although the uncertainty of measurement correlates poorly with the ratio of the areas under the curves, correlates better with the areas over the curves. Therefore, although the uncertainty of measurement has little effect on the diagnostic accuracy of the test, it has a considerably greater effect on its diagnostic inaccuracy.

Reference: T. O. Lim, R. Bakri, Z. Morad, and M. A. Hamid, "Bimodality in Blood Glucose Distribution—Is It Universal?," Diabetes Care,25(12), 2002 pp. 2212-2217.