David Schie, CEO AIStorm Inc, CEO Linear Dimensions.
The hardware industry has a power problem, and it’s not the one most teams are trying to solve.
I’ve watched engineering leaders optimize neural networks, swap accelerators and tune every parameter only to find their devices drain batteries faster than expected. While many leaders think the issue is the models, the underlying architecture is most often the culprit.
Here’s what most power budgets actually look like: Your sensor captures a signal, and that analog data gets converted to digital by analog-to-digital converters (ADCs) that run constantly and burn watts. The digital stream shuffles into memory. The processor wakes up, pulls that data back out, runs inference, writes results back to memory and sends output somewhere else.
Each of those handoffs costs energy. When you add them up, the actual neural network computation accounts for a fraction of total power. The rest? Data movement.
A 2014 Stanford analysis found that moving 64 bits of data off-chip costs roughly 100 to 1,000 times more energy than the arithmetic operation itself. Newer accelerator generations haven’t closed that gap much, since compute logic has gotten several times more efficient while DRAM access energy has stayed about the same.
It’s not the math that kills you, it’s the commute. I’ve seen teams spend months shaving 15% off their model’s compute cost, only to realize they’re still moving the same amount of data through the same pipeline. Even when the model got faster, the battery life didn’t change.
Take memory access. AI workloads don’t burst; they stream. Memory systems designed for occasional reads can’t handle sustained load without burning power. Even a small model becomes undeployable when the memory traffic alone blows your budget.
I’ve spent years building AI directly into imaging silicon for mobile and wearable cameras, and I’ve watched a team shave real time off their model’s compute cost expecting battery life to follow, only to find it barely moved because memory traffic from constant parameter lookups and buffer updates was eating the savings. If you’re only optimizing the model, you can end up shipping a product that still doesn’t last a full day.
Old Assumptions, New Realities
The traditional hardware stack made sense when workloads were intermittent. A sensor would sample occasionally. A processor would wake up, crunch numbers and go back to sleep. Memory was a pit stop, not a highway, and AI killed that rhythm.
Modern AI workloads don’t sleep, but constantly stream. A camera doesn’t just capture frames; it feeds a continuous visual pipeline. A microphone isn’t waiting for a keyword; it’s analyzing audio live. The processor is running inference hundreds of times per second.
I’ve seen this on camera-sensor projects for phones and smart glasses, where features like face recognition and keyword spotting need to run continuously without draining the battery. On one project, the team assumed the neural network would be the hard constraint and found the ADCs running nonstop to feed it were the higher cost.
Rethinking The Pipeline
Rather than focusing on optimizing the old architecture, the teams making real progress are questioning it entirely. What if you didn’t move the data at all? What if you kept it analog longer? What if the computation happened right where the signal originates?
Some teams are pushing computation closer to the sensor. Sony’s IMX500 image sensor runs inference directly on the sensor die, so a frame never has to leave before anything useful happens with it. An independent benchmark shows it delivers the highest compute utilization of the edge AI processors tested. With this model, you’re not just saving power but changing the entire system’s operating model. The trade-off is flexibility: You inherit the sensor vendor’s roadmap, so this fits a stable task more than a model still under active development.
Others are keeping data analog longer, since early digitization is expensive and skipping it for certain operations cuts out a major power drain. Event-based vision sensors, like those from Prophesee, only report a pixel when it changes rather than digitizing a full frame on a clock, dropping idle power into the microwatt range.
AIStorm, where I work, takes a related approach on the compute side, coupling AI processing directly to a sensor’s raw analog output before digitization, keeping always-on power in the single-digit microwatts for imaging and audio. The trade-off is precision and tooling maturity, so this suits tasks that can tolerate approximation more than ones needing bit-exact results.
A few are rethinking memory entirely, folding parameter storage into the compute layer instead of shuttling data between the two. Honda signed a joint development agreement in 2026 to co-design an automotive AI chip built around this approach, precisely because vehicles are so power-constrained. The ecosystem is still young and better suited to teams with a longer runway than a product shipping next quarter. The best practice here is to prototype on conventional hardware first and port once the architecture stabilizes, rather than committing to both at once.
Not every team needs to rewrite their architecture. A large camp is instead squeezing more from the conventional pipeline through better quantization and purpose-built accelerators, which is lower risk and often the right call when power isn’t the binding constraint. The rewrites above earn their complexity only when data movement, not compute, is what’s draining the battery.
Building For Continuous AI
If you’re building AI into a product, whether a wearable monitoring vital signs or a security camera running object detection, picking a chip and a sensor and connecting them with software leaves performance on the table.
By planning for continuous operation from the start, you are more likely to end up with products that run cool enough, last long enough and cost little enough to deploy at scale.
Start by mapping where your power budget goes, compute or movement, before optimizing either one. Then match the approach to the task: Push compute to the sensor for stable workloads, stay analog longer, trading some precision for efficiency or rework memory if your timeline allows for less mature tooling.
The architecture question deserves the same scrutiny teams already give their model choices.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?