Stanford CIS

(Un)Fairness of Risk Scores in Criminal Sentencing

By Danielle Citron on

In a ruling today, the Wisconsin Supreme Court sanctioned the use of COMPAS risk assessment scores about a defendant’s potential recidivism in sentencing, so long as judges receive written warnings about the value of such scores and don’t rely on them exclusively. In that case, Eric Loomis challenged the trial court’s use of COMPAS scores in sentencing on due process grounds. Defendant’s argument boiled down to this: it was unfair to rely on a score whose accuracy cannot be assessed, interrogated, and challenged. Loomis and his attorneys certainly had notice about the score, but how the private company Northpointe came up with it was a mystery. The scoring system is a trade-secret protected black box, as my colleague Frank Pasquale would say.

Before getting into the court’s ruling and concerns raised by this decision, it is important to understand what COMPAS risk scores assess. As the decision explains, risk scores “are intended to predict the general likelihood that those with a similar history of offending are either less likely or more likely to commit another crime following release from custody.” They “do not predict the specific likelihood that a specific defendant will engage in criminal activity.” Instead, they provide “a prediction based on a comparison of information about the individual to a similar data group.” So they supposedly tell us what people situated similarly to the defendant are likely to do when leaving the carceral state. The idea behind the scores is that they provide evidence-based reasons for sentences rather than “ad hoc,” gut-based decisions by human beings beset with cognitive limits and implicit biases.

Ultimately, the defendant’s due process argument did not prevail, and his request for a re-sentencing hearing was denied, because his sentence was based on a host of factors, including but not exclusively the COMPAS score. The Wisconsin Supreme Court emphasized that the COMPAS risk score is partially based on a person’s criminal history that can be verified and thus the score could be challenged on that basis. The Court, however, did not acknowledge that even with that historical data, challenging an overall score is still difficult since no one–crucially including the defense–has any idea how much weight that history has in the overall risk assessment.

Read the full piece at Forbes.