Arun Kumar Elengovan is director, engineering security at Okta.
The transition from AI making recommendations to decision-making and action itself transforms the equation of trust. In the case when AI makes a recommendation, the human is still responsible for analyzing the recommendation and making further decisions based on it. Nevertheless, an autonomous agent is able to make its own decisions, gain access to sensitive information, pay and interact with other agents without any human intervention.
Without the right safeguards, we risk creating agents that have authority to act on our behalf but lack the boundaries required to protect our interests. That could lead to privacy violations, financial losses and, ultimately, a loss of human control.
As I see it, the future of agentic computing should be oriented toward intent-based AI authorization, whereby any action taken autonomously should always be in accordance with the original intent of the human.
Permission Is No Longer Enough
Traditional identity and authorization systems were designed primarily to determine who can access a resource and what operations they’re permitted to perform. Identity and access management, role-based access control and protocols such as OAuth determine whether you are authorized to access information or perform certain actions. That model made sense because the human was doing the work.
Agentic AI introduces a different challenge. An AI agent might have legitimate permission to access financial records or make a payment, yet still use that access in a way the user never intended. Imagine asking an agent to book a rental car. It might also have access that allows it to book your flight or hotel. It may have permission to perform those actions, but that does not mean it should.
The next generation of authorization therefore has to go beyond verifying whether an agent can perform an action. It needs to determine whether that action should be performed in the context of the human’s instructions.
Intent Must Travel With The Agent
Preserving that intent across multiple agents requires a technical architecture. I think of it as a series of layers, with each building off the layers before it.
1. The Human Intent Layer: This is where we capture the user’s objectives, preferences, boundaries and consent.
2. The Intent Credential Layer: This layer converts that intent into a machine-verifiable, constrained authorization credential. Existing technologies such as JWTs could represent scopes and claims in a machine-verifiable format.
3. The Trust Governance Layer: A common policy-enforcement point validates the credential, confirms the action remains within the user’s boundaries and detects conflicts.
4. The Agent Execution Layer: This layer controls how agents perform tasks and delegate authority across systems.
5. The Provenance And Assurance Layer: Provenance means maintaining verifiable records of decisions, delegations and executions. Assurance defines what happens when something goes wrong, including remediation, recovery and potentially financial protection.
If an agent makes a payment that does not reflect the user’s intent, accountability cannot end with the explanation that the AI acted autonomously. There needs to be a mechanism for correcting the outcome.
Agents Must Recognize Conflicts Of Interest
It is also possible that an agent may fulfill the user’s request without necessarily being aligned with his or her interest. For instance, in the case of personal shopping, an agent may choose an insurance package that the user desires to be robust and affordable; however, the platform might have gotten some commission from one particular provider. It means that the agent can fulfill the user’s request without really aligning its choice with the needs of the user.
It opens up another area of risk. Actions that are commercial yet authorized are not in line with the user’s interest. While conventional security has been concerned mostly with unauthorized actions and malicious intentions, agentic security requires the inclusion of detection of manipulation, hidden incentives, biased advice and inter-agent conflicts, among others.
For consequential decisions, agents should provide understandable explanations and seek human approval when conflicts arise and those explanations need to be concise enough for a human to actually use them.
Keep The Human Sovereign
I describe this approach as a “Sovereign Human” architecture. The principle is that AI can act on behalf of humans, but the person originating the request should retain meaningful control over the authority, decisions and consequences. The system should ultimately favor the customer or end user, not the platform, vendor or company operating the agent. That is the sovereign aspect of the model.
As AI becomes more autonomous, the future of AI security should not be measured simply by how effectively humans protect machines or machines protect humans. It should be measured by how reliably an agentic system preserves the intention of the human who originated the request.
Agents should represent the human. They should act as an extension of the human. No matter how many systems stand between instruction and execution, that human intent should remain intact.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?