Gregg Aldana, Global SVP at Appian, advises CIOs, CTOs and business leaders globally on enterprise AI and transformation.
I spend a lot of time with CIOs, CTOs and business leaders talking about AI, and one question keeps surfacing: “Why did our AI bill jump again?” IT can usually explain the mechanics. More users, more agents, more API calls, more tokens. But then somebody asks the harder question: “What did we actually get for it?” That is where the conversation gets more interesting.
I have started calling this the “AI consumption trap.” Companies are getting good at measuring AI consumption, while many still struggle to explain what it is worth. I have seen this problem through generations of enterprise technology. Earlier in my career, while leading large modernization work at the FDIC during the global financial crisis, nobody cared how many servers we were consuming or how many lines of code we had written. The questions were more practical. Could we process the work faster? Could we reduce risk? Could people make better decisions? Could the system handle what the mission demanded of it? I think AI deserves the same standard.
AI Activity Is Not Business Value
Enterprise leaders naturally gravitate toward what is easy to measure. That is not unique to AI. A sales organization can count calls and emails long before it knows whether those activities are producing better pipeline. A factory can add shifts without improving profit. AI has given us a new activity metric in consumption.
That metric is useful for budgeting, but it is not value. A rising token bill may mean waste. It may also mean the organization is successfully using AI at scale. Without knowing what process changed, the number tells you little. In my experience, this is why the question “What is the ROI of AI?” is often too broad to be useful.
AI does not have an ROI in isolation. Claims processing has an ROI. Underwriting has an ROI. Contract review has an ROI. Employee onboarding has an ROI. I encourage executives to start with the process and ask what better means.
If an insurer spends $1 million more on AI but removes $10 million in claims handling costs, shortens cycle time and gives adjusters back thousands of hours, I would not be worried that AI consumption went up. I would want to understand how we could scale what worked. Inexpensive AI that changes no meaningful outcome is still expensive. That is the discipline I think many companies are still building: connecting AI spend to the economics of the work it is supposed to improve.
Not Every Process Needs AI
Another mistake is applying AI where traditional automation is a better fit. If a step should produce the same answer every time, use deterministic automation. It will often be simpler, cheaper and more reliable. AI becomes more interesting when the work involves ambiguity, such as reading messy documents, interpreting language, finding patterns or identifying exceptions.
Underwriting is a good example. An application can arrive with hundreds of pages of medical records, inconsistent formatting and important details buried in unstructured information. A human reviewer may spend hours finding what matters. AI can help surface patterns and exceptions and accelerate that review, but when the decision is consequential, a person should still be accountable for the final call.
I explain this to executives simply. If the work is deterministic, automate it. If it requires interpreting ambiguity, documents, language or exceptions, that is where AI becomes more interesting. When the consequence matters to a customer, employee, patient or citizen, somebody should remain accountable for the decision.
The Operating Model Matters
Another issue is AI sprawl. One team has an agent, another has a co-pilot and a business unit is experimenting with a different model. Each decision may be reasonable on its own. The challenge comes when nobody sees the portfolio.
I believe in decentralized experimentation. I do not think every useful AI idea should wait for a central committee, but leaders still need visibility. When I talk with executives about this, I usually bring the conversation back to three questions. First, where are we using AI—not just which vendors or models but in which business processes? Second, what is it actually changing? Is it improving cycle time, cost per case, error rates, revenue, capacity or customer experience? Third, where do people need to remain accountable, particularly when AI is interpreting ambiguity or influencing a consequential decision?
Once those answers are visible, leaders can make better capital allocation decisions. They can scale what works, stop what does not and govern risk according to the consequence of the decision. What concerns me is not necessarily the amount of AI being deployed. It is that many companies still lack an operating model that lets leadership see where AI is creating value, where it is creating risk and where it is simply creating cost.
The Technology Will Change—The Management Discipline Will Not
Models will continue to improve, agents will become more capable and the economics of AI will continue to change. I have worked through enough technology waves to be skeptical of any transformation measured primarily by how much technology people consume. Technology changes, but management discipline does not.
AI will not tell you which business problem is worth solving. It will not decide what better looks like for your company, and it will not tell you whether a process improvement is worth the investment. That remains a leadership responsibility.
So when someone asks, “What did we actually get for it?” I would want the answer to be something the business recognizes. Faster decisions, lower costs, greater capacity, fewer errors and better customer outcomes are measures executives understand. As AI becomes cheaper and more ubiquitous, simply having access to intelligence will become less differentiating. The advantage will belong to organizations that know where to apply it, what outcome they expect and how to scale what works.
The winners in AI will not be the companies that consume the most intelligence. They will be the companies that become best at turning that intelligence into business value.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?