Tiago Azevedo is CIO at OutSystems, an Agentic Systems Platform built for the enterprise.
For the past few years, the approach to AI adoption among enterprises was more or less unanimous: More is more, and any organization lagging to adopt AI would be left behind. Token costs were a small enough line item that teams could get away with unfettered experimentation, and while AI projects were nascent, the threat of AI sprawl was just a blip on the horizon. Blindly confusing AI use with productivity gains, the industry rushed to tokenmaxxing—and then tokenminimizing, once token costs caught up.
Enterprises are quickly realizing that in the rush to adopt AI by all means necessary, they may have sleepwalked into a governance and cost-benefit crisis. The risks are real. Shadow IT is back with a vengeance: teams deploying public-facing AI apps with no authentication, credentials committed to version control through AI-assisted coding, a single user running up thousands of dollars in model costs in a single day with no guardrail in place. These aren’t hypothetical risks. They’re happening now, inside companies that think they’re doing AI well.
The real winners aren’t the organizations exclusively using the newest or most powerful models. They’ve invested in the infrastructure and operational discipline to avoid runaway AI costs and retain control over model use. Today’s CIOs need to understand the realities of token economics and model choice, as well as the infrastructural shifts needed today, to ensure successful deployments tomorrow.
Flexibility And Control From The Ground Up
It’s not realistic to expect teams to pause and consider which model is best suited to each and every use case—sure, costs would probably even out, but such manual intervention would also negate time savings. The solution is not cultural, but infrastructural. Before scaling AI projects, organizations must invest in technology infrastructure that guarantees model flexibility and governance, and determines the model best suited for each use case, by default.
This wouldn’t be another tool, but a vendor-agnostic inference layer at the very foundation of your AI efforts: the “air traffic control” between every AI user and every AI model in your stack. Before someone at your organization uses an AI model or agent, their request is routed through this gateway, which uses MCP connections and enterprise data fabric to identify the right model for a job, route requests across models, enforce governance controls and track costs per user.
For example, this inference layer would allow organizations to automatically route a simple request to a lighter, cheaper model and reserve the heavy models for tasks that justify them. All usage is logged and attributable, and when a new model ships, teams can seamlessly evaluate and adopt it without starting from scratch. This has been incredibly valuable for our own engineering team. Not only does the inference layer allow flexibility in model choice while carrying over our governance standards, but it creates a unified source of truth for us to track usage patterns and identify missing capabilities, or promote solutions at the corporate level to mitigate duplicative work.
With this kind of control and flexibility by design, organizations like our own can, in real time, evaluate new models and capabilities as they emerge, optimize for cost and performance and adapt to changing regulatory environments without interrupting progress.
Considering what’s at stake—operational continuity, cost control and performance optimization—flexibility is no longer a nice-to-have. It’s a business imperative.
Reining In AI Costs
AI sprawl is the new SaaS sprawl, with an even greater financial impact, as organizations transition from isolated AI pilots to full-scale enterprise productions. It’s a three-way squeeze: The business expects the speed and cost improvements promised by AI, regulators and CFOs raise concerns about governance and runaway AI costs and most AI stacks are fragmented and messy. Still, now is not the time for CIOs and CFOs to point fingers, but rather to establish financial governance over AI.
Visibility and auditability are essential, and with the right agentic infrastructure, teams are halfway there. Your organization should have visibility into which models are being applied to specific business functions and use cases and the value those models are driving.
For a business-critical application that requires unparalleled security standards, for example, a high-cost model may still be worth the spend. The companies that build the capability to route the right model to the right task, paying frontier rates only where frontier capability is genuinely required, will have a structural cost advantage that compounds over time.
Just as organizations pay a premium for experienced, highly-skilled professionals, finance teams should understand that certain use cases will require a greater AI investment. The mindset should be far more complex than “How many tokens did we consume?” Instead, consider the ROI of each token.
Optimizing For The Token Economy
At a time when cost, performance and geopolitical conditions are constantly changing, CIOs need to pair real-time visibility and governance with rigorous oversight and conditions for success. “Keeping up” with the breakneck pace of AI by layering tools on top of tools is just a new type of tech sprawl. The time is now to build AI on an infrastructure designed for long-term performance, visibility and control—or risk letting the model-of-the-week dictate all three.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?