Unit 12: Generative AI and Agents in Institutions

Most of the evidence in this course concerns predictive models: scores that inform a human decision. The deployment decisions most leaders now face are about generative models that draft, summarize, answer, and write code, and about agents that take actions on their own. The six questions from Unit 2 still apply. This unit adds the concepts specific to these systems.

The Jagged Frontier in Practice

Field experiments show large gains on some tasks and losses on others, sometimes for the same people with the same tool (Unit 9). Consultants using GPT-4 did better on tasks inside the model's capability and worse on a task just outside it (Dell'Acqua et al. 2023). Experienced developers took 19% longer on tasks in their own codebases with AI tools, while believing they had been faster (METR 2025). The frontier also moves as models change. A task map from last year may be wrong today.

The Core Economics: Generation Is Cheap, Verification Is Not

Generative AI drives the cost of producing a draft toward zero. The binding constraint becomes the cost of checking it, plus the expected cost of errors that slip through. A task benefits when the time saved in production exceeds the time spent verifying plus the expected cost of undetected errors. That simple inequality explains most successes and most embarrassments.

In Mata v. Avianca (S.D.N.Y. 2023), lawyers were sanctioned after filing a brief that cited court decisions invented by ChatGPT. The technology failed in a predictable way. The process failed because nobody checked citations against a primary source, the one verification step that was cheap to perform.

Reliability and 'Hallucination'

Generative models produce fluent output whether or not it is correct, and their expressed confidence is an imperfect guide to accuracy. Mitigations reduce but do not eliminate errors. The main ones are:

• Grounding answers in retrieved documents and requiring citations to them.
• Constraining outputs to structured formats that can be checked automatically.
• Allowing the system to say 'I don't know' and routing those cases to people.
• Concentrating human review where errors are both costly and detectable, rather than spreading it thinly over everything.

Evaluate on Your Own Tasks

Public benchmarks measure what model developers chose to measure, often on data the models may have seen in training. They are weak evidence about your workload. A small, well-designed internal evaluation is far more informative:

1. Collect a few dozen to a few hundred real, representative cases, including hard and unusual ones.
2. Have experts produce reference answers and define which errors matter and how much.
3. Measure the AI-assisted process against the current process, not against perfection.
4. Re-run the evaluation whenever the model, prompt, or workflow changes — providers update models frequently.

Agents: Delegating Action

Agents combine a generative model with tools: they can search, read files, send messages, edit records, or spend money. This changes the risk profile in three ways. Errors compound across many steps. Actions can be irreversible. And the agent can be manipulated by the content it processes. In 'indirect prompt injection', instructions hidden in a web page, email, or document hijack an AI system that reads them (Greshake et al. 2023). OWASP lists prompt injection as the top risk for applications built on large language models.

Where Human Review Adds Value — and Where It Doesn't

Requiring a person to approve every output feels safe but often isn't. Reviewers facing a high volume of mostly correct output drift into rubber-stamping (automation bias, Unit 11). Review works best when it is targeted at high-cost, checkable cases, when reviewers have the time and information to disagree, and when their catch rate is measured.

There is also a longer-run cost. Many professions train juniors by having them produce first drafts. If AI produces the drafts, institutions must deliberately rebuild how expertise is developed, or they will lose the reviewers they depend on. Evidence from education points in the same direction: unguarded AI help raised practice performance but lowered later learning (Bastani et al. 2025, Unit 6).

Build, Buy, and Lock-In

• Models change frequently. Design workflows so the underlying model can be swapped, and keep your evaluation sets so you can test replacements.
• Contract explicitly for data use, retention, and residency — including whether your data may be used to train the provider's models.
• Weigh hosted APIs (fast, capable, but dependent on one provider) against open-weight models (more control, more operational burden).
• Own the assets that create lasting value: your data, your evaluations, and your redesigned processes (Unit 2's system redesign).

Transparency duties are also arriving. Under Article 50 of the EU AI Act, from August 2026 people must be told when they are interacting with an AI system, and AI-generated or manipulated content must be disclosed or marked (Unit 11).

Key Takeaways

• Map tasks against the jagged frontier with your own evidence; gains and losses can coexist in the same team.
• Value depends on the cost of verification. Deploy where checking is cheap or already happens, and invest in verification where it isn't.
• Build small internal evaluations from real cases and re-run them whenever the model or workflow changes.
• Agents add compounding errors, irreversible actions, and prompt injection. Use least privilege, approvals, limits, logging, and an off-switch.
• Blanket human review degrades into rubber-stamping; target it, resource it, and measure it — and protect how expertise is developed.
• Avoid lock-in: keep models swappable and own your data, evaluations, and processes.

Evidence & Further Reading

• Dell'Acqua, F., McFowland, E., Mollick, E., et al. (2023). Navigating the Jagged Technological Frontier. Harvard Business School Working Paper 24-013.
• METR (2025). Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity. Research report, July 2025.
• Mata v. Avianca, Inc., No. 22-cv-1461 (S.D.N.Y. June 22, 2023) (order imposing sanctions).
• Greshake, K., Abdelnabi, S., Mishra, S., et al. (2023). Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection. Proceedings of the ACM Workshop on Artificial Intelligence and Security (AISec).
• OWASP Foundation. OWASP Top 10 for Large Language Model Applications.
• Bastani, H., Bastani, O., Sungu, A., et al. (2025). Generative AI without guardrails can harm learning: Evidence from high school mathematics. Proceedings of the National Academy of Sciences, 122(26), e2422633122.
• Regulation (EU) 2024/1689 (Artificial Intelligence Act), Article 50.