The United States Supreme Court has set forth a test for determining the constitutional adequacy of the process by which government action adversely affects someone's life, liberty, or property. In Mathews v. Eldridge, the Court required consideration of three factors: (1) the private interest that will be affected by the official action; (2) (a) the risk of an erroneous deprivation of such interest through the procedures used and (b) the probable value, if any, of additional or substitute procedural safeguards; and (3) the government's interest, including the function involved and the fiscal and administrative burdens that the additional or substitute procedural requirement would entail.
This Demonstration attempts to parameterize the Mathews v. Eldridge framework. It does so by considering a situation in which the law bars official action against a person unless their behavior or situation is believed to have fallen below some cutoff level. The actual behavior or situation cannot be perfectly detected, however, at least not without significant cost. Rather, it can be measured imperfectly using different procedural mechanisms, each of which has different costs. The frequent legal issue is which set of procedural mechanisms must the government employ before taking away a constitutionally protected interest.
You select from among 10 datasets, each of which provides information on 100 sample people. The people have three scores (each measured by three procedures) that provide information on the actual behavior in which they engage. They also each have an actual level of behavior that some omniscient being or perfect test could determine. For convenience, the results of all tests and the actual behavior lie between zero and one for all persons. You select for comparison two of the possible procedure sets the government could employ. Each of these procedure sets is a non-empty subset of the set of all three possible procedures. The procedure sets correspond roughly to the second Mathews v. Eldridge factor, since they determine the risk of an erroneous characterization of the person's behavior. You select the cost of type I (false positive) and type II (false negative) errors that result from mistaken determinations as to whether the person's behavior falls short of a cutoff value that you determine using a locator. The magnitude of a type I error may be thought of as parameterizing the first Mathews v. Eldridge component. You likewise choose the cost of running each test, which, when coupled with the cost of a type II error, may be thought of as parameterizing the third Mathews v. Eldridge factor relating to the costs of operating a procedure set.
The Demonstration uses linear regression to determine for each of the two user-selected procedure sets a reasonable mapping between the scores of a person on those tests and their predicted behavior. It then computes "contingency tables" for each such mapping and for varying prediction cutoff scores. The contingency tables classify the people in the dataset according to whether they are "truly positive" (i.e., their behavior falls below the actual cutoff) and according to whether they are "test positive" (i.e., whether the mapping predicts their behavior will fall below the prediction cutoff score). From these contingency tables, the Demonstration computes the total costs of operating the procedure set for each prediction cutoff score.
In this Demonstration, the cutoff level is described numerically by reference to some metric. This might occur where the issue in question is the person's income or assets or their score on some test. The cutoff level can also be described verbally.
The text of Mathews v. Eldridge may be found here.
Snapshot 1: The initial settings of the Demonstration. The minimum total costs of procedure set are less than the minimum total costs of procedure set even though, as shown from the tooltips associated with the minimum points, procedure set is slightly more accurate. Notice that both procedure sets do best when there is a safety margin: the prediction has to be substantially worse than the actual cutoff before the government takes adverse action against the person.
Snapshot 2: The initial settings of the Demonstration modified so that the costs of procedure 3 are zero. Now procedure set is slightly less costly than procedure set .
Snapshot 3: The initial settings of the Demonstration modified so that the costs of procedure 3 are zero and type I error costs are lower and type II error costs are higher. Now procedure set is somewhat better than procedure set but both should employ negative safety margins, that is, authorizing adverse action against the person even when their predicted score is somewhat higher than the actual cutoff.
Snapshot 4: The initial settings of the Demonstration but comparing procedure set with procedure set . Procedure set has lower total minimum costs.
Snapshot 5: The initial settings of the Demonstration except that dataset 8 is used.