How Do You Contain an AI Agent Failure You Can't Prevent?
The article posits that AI agent failures are an accepted reality, with previous parts of the series establishing the concept of "acceptably wrong" and measuring error frequency. The central question shifts from preventing failure to containing its impact, which is presented as a design choice that teams often overlook. The severity of an agent's mistake is not fixed but depends on the power it wields. Illustrative examples highlight this distinction: an agent that drafts an email for human review carries different risks than one that sends it autonomously, or one that suggests a refund versus one that issues it. In both cases, the identical mistake leads to vastly different consequences based on the agent's authority. The article emphasizes that the "blast radius" is set by choosing what an agent is permitted to do, not by hoping it performs correctly. It advises granting an agent the minimum authority required for its task, suggesting it should read before writing and propose before executing. An action exceeding an agent's measured reliability becomes a chosen liability; for instance, if an agent is right "eighty percent of the time," it should not be allowed to move money unsupervised. Capability should align with trust, which is quantifiable.
Deciding an AI agent's authority when it fails determines whether an application can ship and manages potential liabilities for developers.