AI for Decision Makers
Unit 1: What AI Is: Capability, Reliability, and the Jagged Frontier
Leaders make two opposite mistakes about AI. Some treat it as an oracle and hand it authority it has not earned. Others dismiss it as autocomplete and miss both real gains and real new risks. Both mistakes come from the same place: a poor mental model of what these systems are and how their performance behaves. This unit gives you a model that holds up in practice.
Three Kinds of Systems You Will Meet
"AI" covers very different technologies. Most institutional decisions about AI concern one of three kinds of system, and each fails in its own way:
| Kind | What it does | Typical institutional use | Characteristic failure |
|---|---|---|---|
| Predictive models | Map inputs to a score or class (risk, fraud, diagnosis) | Bail risk scores, credit scoring, sepsis alerts, dropout early warning | Quietly wrong on populations unlike the training data; encode the biases of the outcome they were trained to predict |
| Generative models | Produce text, images, code, or summaries from instructions | Drafting, summarizing records, answering staff questions, clinical documentation | Fluent, confident errors ("hallucinations") that are expensive to detect |
| Agents | Generative models that plan multi-step tasks and take actions through tools | Processing cases end to end, operating software, sending messages, making purchases | Errors compound across steps; can be manipulated by content they read; actions may be irreversible |
Most of this course's institutional cases involve predictive models, because that is where a decade of evidence exists. Unit 12 is devoted to generative AI and agents, which are now where most new deployment decisions sit.
Capability Is Jagged, Not a Ladder
It is tempting to place AI on a single scale from "narrow tool" to "human-level". Evidence from the field shows a different picture. In a preregistered experiment with 758 BCG consultants, those given GPT-4 completed 12.2% more tasks, 25.1% faster, at over 40% higher quality on tasks inside the model's capability. On a task deliberately chosen to lie just outside it, they were 19 percentage points less likely to reach the correct answer than consultants working without AI (Dell'Acqua et al. 2023). The researchers called this boundary a "jagged technological frontier": the same system can be excellent at one task and unreliable at a neighbouring task that looks equally hard to a person.
Measured performance can also come from the wrong reasons. Deep-learning models trained to detect pneumonia on chest X-rays performed worse when tested at hospitals other than the ones that supplied their training data (Zech et al. 2018). The models had partly learned signals specific to each hospital rather than to the disease. A model can score well in testing and still fail in your setting. That is a property of how these systems are built, not a rare accident.
An Empirical Field: The Engineering–Science Gap
Michael Jordan (UC Berkeley) describes modern AI as a field that has advanced as engineering faster than as science (Jordan 2019). We can build large learned systems that work impressively. But there is no mature theory that predicts, ahead of time, when they will generalize to new conditions and when they will fail.
| Dimension | What engineering gives us | What the missing science would give us |
|---|---|---|
| Goal | Systems that reach measured performance on chosen tests | Principles that predict behaviour under conditions we haven't tested |
| Evidence | Benchmarks, demos, and validation sets | Guarantees about where performance will hold and where it will break |
| Implication for leaders | Performance claims are local to the conditions they were measured in | Until the science matures, local validation and ongoing monitoring stand in for theory |
Use this idea precisely. Not every AI failure is a mysterious generalization failure. Some have mundane explanations: poor training data, no external validation, commercial overreach (Unit 7 shows a famous example). The gap matters most for large learned models whose internal workings resist inspection. It means a vendor's benchmark tells you what a system did somewhere else. Only evidence from your own context tells you what it will do there.
Calibrated Trust: Two Ways to Get It Wrong
When people work with algorithmic advice, they err in both directions. Automation bias is over-reliance: deferring to the system even when other information says it is wrong (Goddard, Roudsari & Wyatt 2012). Algorithm aversion is under-reliance: abandoning a demonstrably better algorithm after seeing it make a single mistake, while tolerating larger human error (Dietvorst, Simmons & Massey 2015). In institutions, the most common pattern is often a third one, selective adherence. Decision-makers follow the algorithm when it agrees with them and override it when it doesn't, sometimes in systematically biased ways (Unit 5).
Automating Decisions vs. Building Markets
Jordan also argues that some of AI's largest effects come not from imitating human judgment but from building new matching and market systems: recommendation engines, ride-hailing, dynamic pricing. Uber did not automate a dispatcher's judgment; it created a real-time market for rides. This lens shifts the evaluative question. Instead of "Is this system intelligent?", ask "How does it reallocate information, choices, and bargaining power among the people it connects?"
Market-creating systems are not lower risk than systems that automate an existing decision. Their risks are different, and often larger: they reshape incentives for everyone in the market, concentrate information in the platform, and generate system-wide effects no single transaction reveals. Units 8 and 9 examine attention markets and algorithmic management through this lens.
A Moving Frontier
AI capabilities have improved quickly and unevenly, and researchers disagree about how far and how fast current approaches will go. A decision-maker does not need to settle that debate. You need practices that stay sound under either answer. Grant authority task by task, on evidence. Re-evaluate when the system or the task changes. Avoid architectures and contracts that lock in today's assumptions about what the technology can and cannot do.
Key Takeaways
- Distinguish predictive models, generative models, and agents. Each fails in a characteristic way.
- Capability is jagged. Judge reliability task by task on your own data, not by where a system sits on an imagined intelligence ladder.
- Replace 'does it understand?' with: how often is it right here, how would we know when it's wrong, and what do errors cost?
- The engineering–science gap means performance claims are local; validation and monitoring in your context substitute for theory.
- Trust can fail in both directions — automation bias, algorithm aversion, and selective adherence. Calibrated trust is designed, not assumed.
- Market-creating systems carry different, often systemic, risks. They are not a lower-risk category.
Evidence & Further Reading
- Dell'Acqua, F., McFowland, E., Mollick, E., et al. (2023). Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of AI on Knowledge Worker Productivity and Quality. Harvard Business School Working Paper 24-013.
- Zech, J. R., Badgeley, M. A., Liu, M., et al. (2018). Variable generalization performance of a deep learning model to detect pneumonia in chest radiographs. PLOS Medicine, 15(11), e1002683.
- Jordan, M. I. (2019). Artificial Intelligence — The Revolution Hasn't Happened Yet. Harvard Data Science Review, 1(1).
- Goddard, K., Roudsari, A., & Wyatt, J. C. (2012). Automation bias: a systematic review of frequency, effect mediators, and mitigators. Journal of the American Medical Informatics Association, 19(1), 121–127.
- Dietvorst, B. J., Simmons, J. P., & Massey, C. (2015). Algorithm aversion: People erroneously avoid algorithms after seeing them err. Journal of Experimental Psychology: General, 144(1), 114–126.