Predictive Scores and Ultimate Test Passage
It is frequently possible to predict whether a particular student at a school will pass some ultimate examination, such as a professional licensure exam or some exam required to achieve a degree. Thus, if one knows the distribution of predictive characteristics for all the students at a school, it is possible to make some composite estimate as to the percentage of students who will pass an ultimate exam. If one understood this relationship, one might then be able to assess whether a particular school, given the profile of students attending it, is doing a better than average or worse than average job in preparing its students to pass the ultimate examination.[more]
This Demonstration examines the expected relationship between the predictive characteristics of the students at a school and passage of an ultimate examination. You predict the score needed to pass the ultimate examination. You select the difference in the predictive score between students in the percentile at a school and students in the percentile. And you select the standard error of individual predictions of scores on the ultimate examination based on the predictive characteristics. The Demonstration outputs a graphic showing the relationship between the mean predictive score of students at a school and the expected pass rate of those students on an ultimate examination.[less]
The numeric values selected for this Demonstration are drawn from the relationship between LSAT scores in the United States and scores on the bar examination generally required to become licensed as an attorney in the United States. In recent years, the American Bar Association, whose accreditation of a law school is generally required before that school's graduates may become attorneys, has required as part of "Interpretation 301-6" that pass rates on the bar examination achieve a certain threshold. Various law schools have complained that this standard unduly burdens them because of the lower predictive scores of the students who attend them. Yet, to know whether such a school is in fact doing a good job educating its students to pass the bar exam, one should know the shape of the relationship between predictive scores and bar passage rates. This Demonstration suggests that, as a matter of theory, the shape should be somewhat sigmoidal.
Snapshot 1: lowering the score required to pass the ultimate examination effectively moves the curve leftward
Snapshot 2: increasing the range of predictive scores of admitted students "flattens" out the sigmoidal shape into a more linear one
Snapshot 3: increasing the standard error of prediction (i.e., weakening the correlation between predictive scores and ultimate examination scores) also flattens out the sigmoidal shape into a more linear one
Snapshot 4: shows the relationship when no school admits students whose predictors are so low that fewer than half will pass an ultimate examination