Approximating a Judge
A judge's decision rule can be thought of as a mapping from facts to outcomes. And often, it is possible to approximate the facts as a set of binary values and the outcome as a binary value such as "affirmed" or "reversed", or a set of binary values. This Demonstration shows how an investigator can take a set of cases seen by a judge along with the associated outcomes and approximate the rule the judge must be applying.[more]
You select the number of facts the judge considers and select from a large number of possibilities the actual binary rule the judge applies. You select the number of cases the judge decides. You then select the set of facts that the investigator is able to perceive. Thus, while the judge may be deciding a case based on facts 1 through 7, the investigator may only be able to perceive facts 4, 6, and 7.
The Demonstration produces a decision tree showing the algorithm that best predicts the decisions of the judge in the decided cases. You also see associated statistics such as the rule number the judge appears to be using, the number of cases the algorithm correctly predicts, and the percentage of cases the algorithm correctly predicts. You can choose the form in which this decision tree will be presented, such as "BDT" ("Binary Decision Tree") or "IF" ("if-then form"). A key at the top right of the output explains the abbreviations used in the decision tree.
It is interesting to explore the extent to which changing: (a) the cardinality of the set of facts actually used; and (b) the set of perceived facts affects the number of cases the algorithm correctly predicts.[less]
The fact that outcomes can often take on more than two discrete values (, , and , for example) does not pose a significant limitation on the technique described in this Demonstration. In such cases one creates a binary representation of the outcome such as , , and and then uses the techniques described here to develop a rule that best computes each binary digit of the outcomes. Thus, if there are possible outcomes, there are rules that are required.
Facts that are not binary but that have either finite discrete values or can be approximated by finite discrete values likewise do not pose a significant limitation on the techniques described in this Demonstration. One simply takes the binary value of the fact and joins that to the other discrete facts of the case.
The technique described here may sometimes be an improvement over more typical techniques such as linear regression in predicting the behavior of judges if only a low-order combination of independent variables is used in the regression. By way of example, a probit or logit regression whose dependent variables were simply the binary values of the facts would likely have poor explanatory capability where the judge was basing his or her decision on various complex combinations of these variables. An ability to predict the behavior of judges is of great value in commercial cases and useful in advising clients on settlement offers.