Unit 5: Criminal Justice

Unit 5 of 13

Criminal justice was among the first domains to use algorithmic risk assessment at scale, and it has the richest evidence base. It is the clearest place to see the course's central lesson. What an algorithm could achieve on paper, what it achieves in practice, and what it measures in the first place are three different questions.

The Institutional Landscape

Stage	Status quo	Algorithmic tool
Policing	Patrol allocation by commanders' judgment and past calls	Place-based forecasting of where crime will be recorded
Pretrial release	Judge decides release, conditions, or bail, usually after a brief hearing	Pretrial risk scores (e.g. PSA, COMPAS) estimating failure to appear and re-arrest
Sentencing	Judge weighs guidelines, record, and circumstances	Risk scores informing diversion, probation, or sentence length
Parole & supervision	Boards and officers assess readiness and risk	Recidivism scores informing release and supervision intensity

Decision: Pretrial Release

The prediction question is relatively clear: if released, will this person appear in court and avoid a new arrest? The judgment question is harder: how much risk justifies detaining someone who is legally presumed innocent, given that detention itself can cost a job, housing, or custody of children? This is close to a pure prediction problem in the sense of Unit 2. But the cost ratio that sets the threshold (Unit 3) is a profound moral and legal choice.

Pretrial Release Through the Five Stages

Stage	Traditional	AI-Mediated	Risk
🔍Framing	Statutes and local practice define 'flight risk' and 'danger', often loosely	Risk is defined as the probability of failure to appear or re-arrest within a window	Re-arrest measures policing as well as behaviour (see 'The Label Problem')
📊Prediction	Brief hearings with limited information; release rates vary widely across judges	Consistent scores from criminal history, age, and charge	Scores inherit differences in historical enforcement across groups and places
⚖️Decision rule	Each judge applies an implicit, unstated risk tolerance	Score bands map to recommended release conditions	The threshold encodes the value of liberty versus risk; it is often set by default rather than deliberated
🎯Action	Judge orders release, conditions, money bail, or detention	Recommendation presented to the judge, who may follow or override	Selective adherence: overrides can be frequent and unequal across groups
🔄Feedback & adaptation	Judges rarely learn what happened to people they released or detained	Outcomes observed only for those released	Selective labels: we never observe what detained people would have done

What Algorithms Could Do

The human baseline is not neutral. Judges facing similar defendants make very different release decisions, and Arnold, Dobbie, and Hull find significant racial discrimination in bail decisions (2022). Using New York City data, Kleinberg and colleagues estimated what a machine-learning risk prediction could achieve if used to guide release decisions (2018). Holding the jailing rate fixed, crime could fall by up to 24.7%. Holding crime fixed, jail populations could fall by up to 41.9%. Both could be done while reducing racial disparities. The study uses quasi-random assignment of cases to judges to address the selective labels problem. Its results are still simulations of a policy, not observations of one.

Key Insight

Against the real status quo, better prediction can improve liberty, safety, and equity at the same time. That potential is the strongest argument for these tools. It is also conditional on how they are used, as the next section shows.

What Happened in Practice: The Implementation Gap

When Kentucky made pretrial risk assessment mandatory in 2011, it changed how bail was set. But the increase in release was small, and it eroded as judges drifted back to their previous practices (Stevenson 2018). Judges frequently overrode the presumptive release of low- and moderate-risk defendants. Albright found that judges were more likely to override lenient recommendations for Black defendants than for comparable white defendants (2019). In Virginia, sentences tracked the risk scores, but Stevenson and Doleac found no detectable gains in public safety or reductions in incarceration (2024). Judges systematically extended leniency to young defendants even though youth predicts higher risk.

Caution

The algorithm's performance on paper is not the policy's performance in court. Outcomes were driven less by the model than by how judges used it — when they followed it, when they ignored it, and for whom. Evaluate the decision system, not the score.

The Label Problem: Re-arrest Is Not Offending

Risk tools are trained to predict recorded outcomes, usually re-arrest or failure to appear. Re-arrest depends on whether offending is detected and recorded, which depends on where and how police patrol. If enforcement is heavier in some neighbourhoods or against some groups, the label itself over-records their 'risk'. Removing race as an input does not fix this, because the bias sits in the outcome being predicted. Any claim that a tool is 'accurate' or 'fair' is a claim relative to that label (Unit 10).

Case Study: COMPAS

Case Study

Criminal Justice

COMPAS: Fairness, Accuracy, and What Was Measured

In 2016 ProPublica analysed COMPAS scores for defendants in Broward County, Florida. Black defendants who did not reoffend were far more likely to have been labelled high-risk than white defendants who did not reoffend. Northpointe, the developer, responded that the scores were equally predictive across races: a given score corresponded to similar re-arrest rates for Black and white defendants.

Metric (ProPublica analysis)	Black defendants	White defendants
False positive rate (labelled higher risk, not re-arrested)	44.9%	23.5%
False negative rate (labelled lower risk, re-arrested)	28.0%	47.7%

Source: Angwin et al. (2016), ProPublica analysis of Broward County data

Two years later, Dressel and Farid showed that COMPAS's overall accuracy on the same population (65.2%) was matched by a logistic regression using only age and number of prior convictions (66.8%), and by the pooled predictions of untrained online volunteers (67.0%). They measured recidivism as re-arrest within two years and noted explicitly that this is not a direct measure of reoffending (2018).

Key Insight

The COMPAS debate is often taught as 'both sides were mathematically right, so fairness is just a value choice.' That is half the lesson. The other half is that the dispute rests on a contested measurement (re-arrest), and that the proprietary tool offered no accuracy advantage over a transparent two-variable rule.

Lessons

Both claims can be true at once; when base rates differ, equal error rates and equal predictive value cannot both hold (Unit 10)
Both claims are measured against re-arrest, so neither settles whether the tool is fair with respect to actual offending
A two-variable model and untrained volunteers matched COMPAS's accuracy: complexity and secrecy bought little
Which fairness property to prioritize is a policy choice that belongs to accountable institutions, not to a vendor

Opportunity Cost: Prediction vs. Prevention

Much pretrial 'risk' is failure to appear, and some of it is cheap to prevent rather than merely predict. In New York City, a redesigned summons form and text-message reminders cut failures to appear by 13–21% (Fishbane, Ouss & Shah 2020). An earlier experiment in Baltimore found that defendants who had a lawyer at their bail hearing were substantially more likely to be released on their own recognizance (Colbert, Paternoster & Bushway 2002).

Illustration (hypothetical numbers): a county weighing a multi-year risk-assessment contract should price it against these alternatives with the same budget, and against the status quo. The point is not that risk tools are worse. Prediction and intervention answer different questions (Unit 3), and the best portfolio may combine a simple, transparent risk rule with cheap interventions that reduce the risks it predicts.

Predictive Policing and Feedback Loops

Place-based forecasting directs patrols to where crime has been recorded. More patrols there record more crime there, which feeds the next forecast. Simulations using Oakland drug-arrest data show how this loop can concentrate enforcement in already heavily policed neighbourhoods, even when underlying drug use is spread more evenly (Lum & Isaac 2016). Formal models show the loop can 'run away' unless the system corrects for its own influence on the data (Ensign et al. 2018). This is interdependence (Unit 4) in its purest form: the system's actions generate its training data.

Key Takeaways

Compare risk tools with the real baseline: judges vary widely and show measurable bias. Better prediction could improve safety, liberty, and equity at once.
In practice, outcomes depend on adherence. Kentucky and Virginia show small, fading, and sometimes unequal effects driven by how judges used the scores.
Re-arrest is a biased label for offending; fairness and accuracy claims are only as good as the outcome they are measured against.
COMPAS illustrates the fairness impossibility results, a contested label, and a proprietary tool no more accurate than a two-variable rule.
Some predicted risk is cheaper to prevent than to predict. Weigh prediction against interventions like reminders and counsel at first appearance.
Place-based policing forecasts can create self-reinforcing feedback loops.

Evidence & Further Reading

Kleinberg, J., Lakkaraju, H., Leskovec, J., Ludwig, J., & Mullainathan, S. (2018). Human Decisions and Machine Predictions. Quarterly Journal of Economics, 133(1), 237–293.
Arnold, D., Dobbie, W., & Hull, P. (2022). Measuring Racial Discrimination in Bail Decisions. American Economic Review, 112(9), 2992–3038.
Stevenson, M. (2018). Assessing Risk Assessment in Action. Minnesota Law Review, 103, 303.
Albright, A. (2019). If You Give a Judge a Risk Score: Evidence from Kentucky Bail Decisions. Working paper.
Stevenson, M. T., & Doleac, J. L. (2024). Algorithmic Risk Assessment in the Hands of Humans. American Economic Journal: Economic Policy, 16(4), 382–414.
Angwin, J., Larson, J., Mattu, S., & Kirchner, L. (2016). Machine Bias. ProPublica.
Dressel, J., & Farid, H. (2018). The accuracy, fairness, and limits of predicting recidivism. Science Advances, 4(1), eaao5580.
Fishbane, A., Ouss, A., & Shah, A. K. (2020). Behavioral nudges reduce failure to appear for court. Science, 370(6517), eabb6591.
Colbert, D. L., Paternoster, R., & Bushway, S. (2002). Do Attorneys Really Matter? The Empirical and Legal Case for the Right of Counsel at Bail. Cardozo Law Review, 23, 1719.
Lum, K., & Isaac, W. (2016). To predict and serve? Significance, 13(5), 14–19.
Ensign, D., Friedler, S. A., Neville, S., Scheidegger, C., & Venkatasubramanian, S. (2018). Runaway Feedback Loops in Predictive Policing. Proceedings of FAT* 2018, PMLR 81, 160–171.