Iri Trashanski, Chief Strategy Officer at Ceva, is shaping the future of the Smart Edge with extensive experience across tech sectors.
Artificial intelligence is entering a new phase where compute demand is growing faster than centralized infrastructure can efficiently support.
The rise of agentic workflows, where systems reason, plan and act across multiple steps, is dramatically increasing the amount of tokens and compute resources required to deliver meaningful outcomes. Unlike traditional AI systems that generate a single output, agentic AI systems can operate continuously across complex, multistep tasks.
In an industrial environment, for example, an agentic AI system might detect abnormal sensor behavior, determine whether the issue requires immediate action, shut down equipment locally to prevent failure and escalate only critical events to the cloud. Each of those actions can trigger multiple cycles of inference, decision-making and refinement.
As enterprises begin deploying fleets of AI agents to handle workflows with minimal human intervention, compute demand will no longer grow linearly. I believe it will compound. And ultimately, this growth is beginning to expose the limits of centralized AI infrastructure.
Centralized AI Is Hitting Structural Limits
Cloud platforms that powered the first wave of AI are facing growing pressure from rising energy demands, infrastructure constraints and the cost of supporting increasingly compute-intensive AI workloads. What once made centralization efficient is increasingly becoming a bottleneck.
As agentic workloads become more complex, the amount of compute required per user interaction can increase significantly, placing additional strain on AI infrastructure and operating economics.
A factory system that needs to shut down failing equipment in real time cannot afford to wait for cloud latency. As AI moves deeper into physical environments, responsiveness will become just as important as raw compute scale.
The next phase of AI requires a different model, one that distributes intelligence across the cloud and the device, assigning workloads based on what each layer does best.
Why Hybrid Intelligence Is Emerging
A hybrid intelligence approach is emerging as the practical path forward. In this approach, AI workloads can be dynamically balanced between on-device processing and centralized compute. Physical AI tasks and lightweight agents can run locally, close to where data is generated, while more complex reasoning and coordination can remain in the cloud.
This shift is driven by practical constraints including latency, privacy, bandwidth, energy consumption and cost. Processing data locally can reduce dependence on constrained cloud infrastructure and allow systems to scale more efficiently.
As agentic AI moves into real-world environments, from consumer devices to industrial systems, I believe this hybrid model will become essential. Devices need to sense, interpret and act in real time, often without persistent connectivity.
Real-World AI Is Already Moving To The Edge
In hearables, on-device AI already adapts noise cancellation and audio processing based on the user’s environment, enabling real-time responsiveness without constant cloud connectivity. These systems can learn patterns and operate without relying on constant cloud interaction.
In industrial IoT, intelligence is increasingly moving closer to where data is generated. Edge-based agents can pre-process data, filter signals and even infer intent before sending anything upstream. In many cases, the cloud only sees the result of that decision, not the raw data.
There is also a growing role for what can be thought of as “wake-up” agents. These ultra-low-power models can run continuously, acting as intelligent filters that determine when a task requires more advanced compute and when it can be handled locally.
Across these examples, the same pattern is emerging: Intelligence is moving closer to the source of data. Decisions are made earlier. Cloud resources are used more selectively. This is the foundation of physical AI. It depends on distributing intelligence across the edge and the cloud in a way that is both efficient and scalable.
Not Every AI Task Needs A Massive Model
One of the biggest misconceptions in AI is that every problem requires a large model. In practice, most real-world AI tasks do not require massive models. They require fast, efficient, purpose-built inference operating within tight power and latency constraints.
A device does not need a large-scale model to detect a pattern, adjust a system or trigger an action. It needs the right model running in the right place. Instead of scaling everything up in centralized data centers, the industry needs to scale intelligence out across billions of connected devices. Smaller models can handle real-time, high-frequency decisions at the edge, while larger models can be reserved for tasks that truly require them.
I believe this shift will change the economics of AI infrastructure.
The Next AI Infrastructure Layer
This transition is driving a new wave of investment, not just in data centers but also inside the devices themselves.
Neural processing units (NPUs) are increasingly being integrated into PCs, smartphones, automotive systems and edge devices to enable AI inference directly on the device. NPUs can enable efficient, low-power AI inference directly on the device, allowing more workloads to be handled locally.
As these capabilities scale across consumer, industrial and infrastructure systems, they can create a distributed layer of intelligence that complements the cloud. That distributed layer is what makes hybrid AI viable at scale. When intelligence is distributed, companies are no longer dependent on massive centralized infrastructure to build AI-driven products. They can reduce cost, improve performance and design systems that are faster, more responsive and more resilient.
The Companies That Scale AI Efficiently Will Win
The cloud will remain essential, but it will no longer be the sole center of AI compute. As agentic systems scale, the limitations of a cloud-only model will become harder to ignore.
The companies that scale successfully will not necessarily be the ones with access to the most centralized compute. They will be the ones that distribute intelligence most efficiently. They will place the right workloads at the edge, use the cloud where it adds value and build systems that treat intelligence as distributed from the start.
Agentic AI does not just increase demand for compute. It also changes where and how that compute must happen.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?
