AI for Decision Makers
Unit 2: Prediction, Judgment, and Who Decides
Every institutional decision — whether to release a defendant, approve a loan, escalate a patient, or flag a student — combines an estimate of what will happen with a view about what matters. AI changes the cost and the location of the first part. That shift, more than any feature of the technology, is what reshapes institutions.
Cheap Prediction Raises the Value of Judgment
Ajay Agrawal, Joshua Gans, and Avi Goldfarb frame modern AI as a drop in the cost of prediction: filling in missing information from the information you have (Agrawal, Gans & Goldfarb 2018). Basic economics follows. When a good gets cheaper, we use more of it, and the value of its complements rises. Prediction's main complement is judgment: deciding how much each possible outcome matters, which errors are tolerable, and what to do. Human prediction, its substitute, loses value.
A common simplification — "AI can predict; it cannot judge" — is not what this argument says, and it is not true as a statement about capability. Modern models produce value-laden recommendations all the time. Large language models are trained in part on human preference judgments. The defensible claim is about authority, not ability. Value trade-offs that affect people's rights and welfare should be set by people who can be held accountable for them. They should be stated openly, not left implicit in a model.
Judgment Doesn't Disappear — It Moves
When a risk score and a threshold replace case-by-case decisions, the value judgments do not vanish. They relocate upstream, into three choices:
- The target: what outcome the model predicts (re-arrest or reoffending? healthcare spending or health need?).
- The objective: what the system is optimized for, and which errors are penalized more.
- The decision rule: the threshold or policy that turns a score into an action.
These choices are made once, often by an analyst or vendor, and then applied to thousands of cases. That changes who exercises judgment, how visible it is, and how hard it is to contest. The governance task is to pull these choices back into the open and assign them to accountable owners. A frontline professional who can only accept or override a score has much less real authority than the person who chose the target and the threshold.
The Five Stages of a Decision
A structured decision passes through five stages. AI can enter at any of them, and the risks differ at each. The 'traditional' column below is deliberately realistic: human decision processes have their own well-documented failure modes, and AI should be compared with them, not with an ideal.
The Five-Stage Decision Framework
| Stage | Traditional | AI-Mediated | Risk |
|---|---|---|---|
| 🔍Framing | Experts define the problem implicitly; definitions vary across people and offices | The problem is fixed by the choice of target variable, available data, and model design | A convenient but invalid target (e.g. spending as a proxy for need) is then optimized precisely |
| 📊Prediction | Intuitive estimates; inconsistent across decision-makers and over time | Consistent probability estimates learned from historical data | Estimates inherit gaps and biases in the historical outcome data, and may not transfer to new settings |
| ⚖️Decision rule | Trade-offs weighed case by case, rarely stated, often noisy | Trade-offs encoded once in an objective and a threshold, then applied at scale | Value choices become invisible and are made by whoever configured the system |
| 🎯Action | A person chooses and carries out the action | The system recommends, defaults, or executes; humans may accept, override, or ignore it | Selective adherence: people follow the system when it agrees with them and override it when it doesn't |
| 🔄Feedback & adaptation | Slow, informal learning; outcomes of rejected options rarely observed | Model retrained on new outcomes; people and organizations adapt to the system | Selective labels, feedback loops, and gaming change the very data and behaviour the system relies on |
Prediction Problem or Causal Problem?
Kleinberg, Ludwig, Mullainathan, and Obermeyer illustrate a distinction most AI proposals skip (2015). Whether to carry an umbrella is a pure prediction problem: you only need to know whether it will rain. Whether a rain dance works is a causal problem: you need to know what happens if you intervene. Pretrial release decisions are close to the umbrella case: what matters most is predicting the person's behaviour if released. Deciding which students to enrol in a mentoring programme is closer to the rain dance. What matters is who would benefit from the programme, which is not the same as who is most at risk. Unit 3 shows why that difference can reverse which people a system should select.
Trade-offs — Relative to What?
Three tensions recur in this course. Each is real at the frontier of what a well-designed system can achieve. Each is often overstated when the comparison is a noisy status quo rather than an idealized human.
| Tension | Where it is real | Where it is overstated |
|---|---|---|
| Accuracy vs. fairness | With different base rates across groups, no score can satisfy every statistical fairness criterion at once (Unit 10) | Relative to biased human decisions, a better prediction can reduce disparities and errors at the same time (Kleinberg et al. 2018) |
| Efficiency vs. accountability | Automated speed can outrun review, and opaque models resist explanation | A written, inspectable decision rule can be more accountable than undocumented individual discretion |
| Consistency vs. context | Some relevant circumstances are invisible to a model | Much of what passes for 'context' in human decisions is noise. Treating like cases alike is itself a core fairness value (Kahneman, Sibony & Sunstein 2021) |
Point Solutions vs. System Redesign
In their later work, Agrawal, Gans, and Goldfarb observe that dropping a prediction into an unchanged process — a 'point solution' — usually delivers little (2022). The large gains come when an organization redesigns the decision system around better prediction: who decides, at what stage, with what default, and with what feedback. That redesign is also why AI adoption is contentious. It moves decision rights and changes whose expertise counts. Expect resistance, and expect the gap between a model's potential and an institution's results to be explained largely by implementation (Unit 5).
A related distinction from Unit 1: some systems automate an existing decision; others create new decisions or markets that did not exist before. Ask early which you are dealing with, because the second kind changes the behaviour of everyone around it.
Six Questions to Carry Through the Course
- Is this a prediction problem, or does the decision depend on a causal effect?
- What exactly is being predicted, and is that label a valid measure of what we care about?
- How does a prediction become a decision — what threshold, what costs, and who set them?
- Compared with what? The real status quo and simple alternatives, not an ideal.
- How will people and organizations respond — adherence, overrides, gaming, feedback loops?
- Who is accountable, and how would we know if the system were failing?
Key Takeaways
- AI lowers the cost of prediction, which raises the value of judgment — the claim is about economics and authority, not about what machines 'cannot' do.
- Judgment relocates into the target, the objective, and the threshold. Make those choices explicit and give them accountable owners.
- Separate prediction problems from causal problems; risk is not the same as benefit from intervention.
- Evaluate trade-offs against the real status quo. Consistency is a fairness value, and Pareto improvements over noisy human decisions are possible.
- Most value — and most conflict — comes from redesigning decision systems, not from inserting a model into an unchanged process.
Evidence & Further Reading
- Agrawal, A., Gans, J., & Goldfarb, A. (2018). Prediction Machines: The Simple Economics of Artificial Intelligence. Harvard Business Review Press.
- Agrawal, A., Gans, J., & Goldfarb, A. (2022). Power and Prediction: The Disruptive Economics of Artificial Intelligence. Harvard Business Review Press.
- Kleinberg, J., Ludwig, J., Mullainathan, S., & Obermeyer, Z. (2015). Prediction Policy Problems. American Economic Review, 105(5), 491–495.
- Kleinberg, J., Lakkaraju, H., Leskovec, J., Ludwig, J., & Mullainathan, S. (2018). Human Decisions and Machine Predictions. Quarterly Journal of Economics, 133(1), 237–293.
- Kahneman, D., Sibony, O., & Sunstein, C. R. (2021). Noise: A Flaw in Human Judgment. Little, Brown Spark.