Daniel Fallmann is founder and CEO of Mindbreeze, a leader in enterprise search, applied artificial intelligence and knowledge management.
Two words are starting to dominate enterprise AI usage: with review.
The agent will open the service ticket with review. Update the record, issue the credit, adjust the order, with review. Those two words are why the risk committee signed off. They are also, in most cases, why the project will not produce the promised return.
Here is the uncomfortable arithmetic. Automation only creates value when checking the work costs meaningfully less than doing it. If a person must open the system, reconstruct the context and confirm each action the agent took, you have not removed labor. You have converted it from execution into verification and moved it to a different desk. Verification is expensive not because humans are cautious but because they are uncertain.
A study published in June examined what happens when agent guardrails escalate decisions to a human. Oversight capacity is finite and depletes. By the 300th approval of a benign action, a reviewer is fatigued and primed to keep clicking approve, so a dangerous action buried in that stream gets waved through. A guard who escalates only five times a day gets a reviewer who reads carefully. The system with more oversight produced the worst outcome because its trivia escalations drew from the same finite pool of attention the serious matter needed.
Attention is not a policy setting; it is a consumable. Every low-stakes approval spends down a budget you will need later, for the one decision that actually mattered.
The Reviewer Who Is Present But Powerless
A paper in AI and Ethics argues that current approaches fail in one of two ways: They strip agents of the autonomy that made them worth building, or they reduce the human to a rubber stamp who signs off on plausible outputs without interrogating the reasoning underneath. The authors separate two kinds of agency: 1) the system holds operative agency and executes, or 2) the human holds evaluative agency—the capacity to verify, steer and override.
Most enterprises have deployed the first and assumed the second, and field research on how people actually oversee software agents suggests the assumption is doing a lot of unearned work. A human who can understand what the system did in time to act on it is a control; a human merely sitting near it is not. Only one is on your org chart.
The Regulator Has Already Named This Problem
The law has reached the same conclusion. Article 14 of the AI Act requires high-risk systems to be designed so a person can effectively oversee them in use, not merely in principle. It obliges providers to keep the overseer aware of the tendency to over-rely on the output, naming automation bias explicitly as a design consideration. European law now holds that a human rubber-stamping an AI recommendation is not oversight and that the burden of preventing that failure sits with those who built and deployed the system.
I would resist filing this under “later.” In June, the Council of the European Union gave final approval to the Digital Omnibus on AI, moving the high-risk obligations from this August to December 2, 2027, for stand-alone systems. The relief was audible, and misread. Sixteen months is two budget cycles, and effective oversight is an architectural property decided at design time, not retrofitted in 2027 onto systems being specified this quarter. Only the deadline moved, not the requirement. And the “with review” line in your business case is about to become a legal claim you may have to substantiate.
What Actually Works
The organizations getting real returns aren’t the ones with the most autonomous agents or the most review gates. They stopped treating oversight as a uniform policy and started treating it as an allocation problem. Four principles do most of the work.
Tier by consequence, not convenience. A blanket review policy fails both ways: It throttles harmless work and exhausts the reviewer before consequential work arrives. The teams that recover from an overwhelmed queue all did the same thing: They mapped actions to consequence tiers before automating.
Reduce uncertainty before reducing oversight. Organizations often try to minimize review by improving models. The larger gains usually come from improving the quality, governance and accessibility of the enterprise knowledge those systems rely on. Better context produces better decisions, which reduces the need for verification. We watched teams buy more capable models to reclaim reviewer time and gain nothing because the hesitation was never about fluency.
Design actions to be verifiable, not merely reviewable. There is a large difference between an agent that produces a paragraph a human must read and second-guess and one that produces a structured, reversible action a human can confirm at a glance. The first consumes attention; the second barely touches it. Reducing the cost of verification is the highest-leverage design decision available, and it is almost always treated as an afterthought. What convinced me was seeing which outputs reviewers actually trusted: always the bounded, reversible ones, never the free-text paragraph.
Instrument the reviewer, not just the model: escalation rate, override rate, time per approval. If override rate collapses toward zero over a quarter, that is not the agent getting better. That is oversight quietly turning into theater, the leading indicator of the incident you have not had yet. The signal that reshaped my thinking was override rate falling to zero, which every time meant the reviewer had quietly disengaged.
The Return Was Never About Capability
The prevailing question in enterprise AI is how much a system can do on its own. It is the wrong measure. The return is not how much work an agent performs; it is how much of that work a human does not have to check. Capability without verifiability only moves labor around.
The defining challenge is becoming less about model capability than about enterprise knowledge. Spend that budget deliberately, or the system will spend it for you.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?