Giles Whiting is CEO of Decisions, a global AI orchestration and process automation company.
Most enterprise AI pilots start in a similar way. A small team launches a promising use case, watches it work and checks what it costs to run. The number is trivial. The technology appears powerful and inexpensive.
Then the use case reaches production. Volume climbs, the invoice becomes material and the architecture is already difficult to change. What looked cheap in a pilot can become an expensive design choice at scale.
This is not simply a token pricing problem. In its May 2026 report, “Taming the AI Cost Curve: AI Cost Optimization Strategies,” Gartner observed the erosion of expected AI value through cost creep and what it calls a silent “token tax.” The more important question is what is creating that cost in the first place.
Often, the answer is architecture. Organizations are using probabilistic models to repeatedly make decisions that are already known.
Paying To Rethink What You Already Know
Language models are valuable when a task requires interpretation, synthesis or judgment. The problem begins when the model is also asked to manage routine control flow. It decides what comes next—routes requests, checks conditions and applies established business logic every time a process runs.
That can look elegant in a demonstration because one model appears to handle the entire flow. In production, however, the organization may be paying inference costs to answer the same fixed questions thousands or millions of times.
The cost can also grow as workflows become more complex. In some architectures, a model repeatedly rereads prior context to determine the next action. The longer the process runs, the more context it may need to process. That means additional steps can increase cost for reasons that have little to do with new reasoning and much more to do with repeatedly reconstructing the state of the process.
Think of this as an orchestration tax. It is not a separate line item. It is buried inside AI consumption, mixed together with the inference that is actually creating value.
Consider a claims-processing scenario. An insurer handling thousands of claims may be tempted to let an AI model determine the routing for every claim: Does it qualify for fast-track approval? Does it require human review? Should it trigger additional fraud checks?
But much of that logic may already be known. Eligibility requirements, thresholds and escalation paths can be defined as deterministic rules that produce the same result each time the same conditions are met. The model can then be reserved for the work that actually requires interpretation, such as reading an adjuster’s unstructured notes and extracting details that predefined rules cannot easily anticipate.
In that architecture, deterministic logic handles the repeatable decisions, while the model is invoked only where judgment or interpretation adds value.
Why Reasoning Is Not Orchestration
The architectural distinction is simple but important.
Reasoning involves tasks such as summarizing unstructured information, interpreting ambiguous inputs and handling cases that cannot be fully anticipated. These tasks are probabilistic by nature.
Orchestration involves sequencing, routing, policy enforcement and other known control logic. When the path is predictable, it can be defined explicitly rather than regenerated through inference.
A useful audit is to examine every step in an AI workflow and ask a basic question: Does this step require genuine judgment, or is it a known decision being treated like a reasoning problem?
That distinction affects more than cost. When control flow is delegated to a probabilistic model, behavioral predictability can decline as well. The same input may not always follow the same path unless governing logic is defined outside the model.
That turns an AI cost problem into a governance problem. Enterprises need to be able to explain what happened, reproduce decisions, enforce policies and determine when a human should review an action. Known business logic is easier to audit when it is represented explicitly.
Designing For Predictability
Technology leaders can apply five practical tests to AI workflows:
• Estimate cost at production volume. Pilot economics reveal very little if transaction counts and model calls will increase dramatically at scale.
• Separate reasoning from control flow. Use models where ambiguity requires them. Use explicit rules, workflows or state machines where the answer should be consistent.
• Reuse stable results. If the same input reliably produces the same output, consider whether that result should be stored rather than generated again.
• Promote settled patterns into explicit logic. If a recurring model decision has become predictable enough to describe consistently, evaluate whether it still needs to be a model call at runtime.
• Treat predictability as an architectural requirement. Cost, behavior and governance should be considered alongside accuracy and latency when designing AI systems.
The goal is not to minimize the use of AI. It is to use AI where its strengths create value. Models are powerful precisely because they can handle uncertainty and ambiguity. Asking them to repeatedly perform work that is already known wastes that strength and can make systems harder to govern.
The organizations that scale AI most effectively may not be the ones that negotiate the lowest token price. They will be the ones that know which decisions require intelligence and which should simply be made once, governed explicitly and reused.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?