“We may have to pace the rate of AI development,” OpenAI’s Sam Altman warned this week. Doing so might “give ourselves enough time for society to harden around these new capability levels.” He’s not wrong. But this seriously underplays the threat.
The imminent surge in AI agents is unprecedented. From a few tens of million to more than a billion in the next three years. We are not ready for this. You would not give your teenager access to your passwords and credit cards, but we are preparing to give our agents exactly that. The pace of this change is a high-risk, high-stakes game. There are not yet agreed rules or appropriate guardrails.
The idea is simple. Why do the hard work to book a vacation — the research, the comparisons, the review checking — when one of your agents can do it for you? Now you can have AI do the legwork, but you’ll check for yourself before you book. Does that flight really exist? Does it really land at that airport at that time? Is that hotel really a bargain?
That will all change. Your agent will run the full process, all the way to booking. Millions of agents will fight for the same rooms, flights, and discounts. No human will be able to compete.
But the real change will come to enterprises, not individuals. They will deploy agents acting with delegated corporate authority. Those agents may have verifiable identities — but not independently verifiable authority for every action. Gartner’s Max Goss calls this “an ungoverned sprawl of agents.”
These risks are not theoretical. Altman was responding to OpenAI’s disclosure that models running an exploitation benchmark without production safeguards found a zero-day vulnerability, gained internet access, and reached Hugging Face’s production infrastructure to pursue their objective. OpenAI called it “an unprecedented cyber incident.”
In a separate, earlier incident, an experimental agent called ROME reportedly tried to mine cryptocurrency without its human handlers ever instructing it to do so.
You can see the problem. Remember, unlike your teenager breaking curfew and sneaking out of the house, this all happens at AI pace and AI scale. You will not know until it’s too late.
In the old world we’re leaving behind, we would introduce stronger permissions and monitoring. We’d never trust, always verify. But that misses the point. What exactly are we being asked to verify? Who exactly are we being asked to trust?
Zero Trust is about to battle hyper-scale agentic AI. And unless and until something changes, Zero Trust cannot win on identity alone. Agents exist virtually and ephemerally. We’re now trying to nail down identity: can I bind my agents to my identity? But that also misses the point. Identity is not authority. Association is not authorization.
The industry’s early answer is just more identity — agent IDs, certificates and tokens. All are necessary. None independently prove authority. Whose token was issued, on what basis and for what purpose? We don’t yet have a better answer. We need to find one fast.
Humans are now trying to invent guardrails for a world that almost all humans do not yet fully understand. That world is moving at a technological pace not seen before. Zero Trust evolved for a reason. Yes, the OpenAI debacle is a wake-up call. Yes, it’s a call to action. But it’s also an admission that the best-practice cybersecurity principles that have evolved over decades are about to be stress-tested beyond the point of failure.
Zero Trust is not dead. But identity is no longer enough. Security will underpin the AI economy, but trust will define it. Authority must be independently verifiable, revocable, and time-limited. It must be rooted in something neither the agent nor its operating platform controls. And it must be checked continuously as the agent acts, transacts, and delegates. That’s the wake-up call.
So, who owns trust? Nobody should. If the same platform that issues an agent’s identity and grants its authority also provides proof that it remains trustworthy, then it’s marking its own homework. The answer is independent and continuous trust signals from authoritative organizations, devices, communications networks and other independently verifiable anchors.
While we’re not approaching the end of Zero Trust, we are approaching the end of Zero Trust as we know it. It’s no longer a case of who’s requesting access. Instead, it’s who authorized it, for what purpose, and over what time period. We do not yet know what this trust layer will look like. But those billion-plus agents will not wait for us to work it out.