Michelle Drolet is CEO of Towerwall, a specialized cybersecurity firm focused on proactive cyber preparedness and compliance services.
Lucy in finance has a new assistant. Let’s call her “AI Lucy”: a digital version of herself that works with her logins, permissions and access to company data.
AI Lucy has made Lucy’s job a lot easier, sorting the inbox, updating the CRM, pulling figures from databases and drafting replies to suppliers, all without Lucy having to sweat the details.
This new way of working isn’t unique to our fictional Lucy, and what she and many real business leaders overlook is the risk. AI Lucy can log in to company systems and act on Lucy’s behalf with very little human oversight.
Most discussions on the risk of AI think narrowly in terms of data leaking from employees pasting sensitive information into a chatbot. AI agent risk derives in part from these autonomous agents acting as privileged digital identities, each with their own network of automated permissions.
Following The Money
Just like employees, AI agents have identities of their own. These are in the form of logins and the authority to act on a user’s behalf. Security vendors have taken note, acquiring AI companies designed to protect identities and govern AI workplace use. One of the world’s largest security vendors paid roughly $25 billion for a leading identity security platform, citing identities, software and AI alike as the main vector by which attackers enter systems.
And it doesn’t end there. A major endpoint security company acquired a startup that monitors how employees use AI tools, while a major player in cybersecurity and compliance acquired a company that monitors how AI tools and agents are used at work and blocks risky activity in real time.
Think of these deals as leading indicators of the kind of threats security teams are most worried about. The message being conveyed is that it is “identity” that rigorously defines what needs protection.
Why Attackers Can’t Help But Love AI Lucy
AI Lucy is the newfound love of attackers because of how these agents operate. Traditional login and access tools were designed for end users to sign in throughout the day and work at human speed. They’re asked to verify themselves. Automated software accounts operate on fixed rules, but AI Lucy has agency. This means it can decide the next course of action without human assistance, transacting thousands of times per day with no oversight.
AI Lucy has also been invested with a boatload of permissions to accomplish its goals. Circling back to Lucy in finance, she likes her assistant so much that she has given it access to her calendar as well, the shared drive, her accounting and expense tools, and all of her access rights. While Lucy doesn’t know it yet, this is slowly but surely setting the stage for privilege creep. AI Lucy is quietly collecting more access than it actually needs.
This makes AI Lucy valuable to attackers through a prompt injection, for example, where hidden instructions trick the AI model into misbehaving. AI Lucy follows the prompt instructions and proceeds to change the payment details on an invoice. Attackers don’t need AI Lucy’s login; they simply trick it and use it as a proxy. Gartner predicts that in two years, AI agent abuse will account for 25% of enterprise breaches.
Of the many businesses I speak with weekly, too few are monitoring the behavior of AI identities. Typically, the actions of compromised agents do not show up in security alerts. If a payment goes to a fraudulent account, finance may fix it as a one-off mistake, but the security team rarely hears about it.
Not Just An Enterprise-Level Problem
AI tools are being deployed by companies irrespective of size. These tools should be kept in check. This includes a multipronged approach consisting of the right security framework, governance rules and trained talent. Large enterprises typically have data security tools and formal risk programs that can minimize AI risk, but many midsize firms do not. They’re often unaware of where their sensitive data resides and which AI identities can access this data.
A Road Map For AI Governance
Employees are using AI tools, whether approved or unauthorized. The question for the C-suite isn’t about AI approval, but governance. The wrong strategy is blocking or allowing AI tools at the gate.
The need of the hour is implementing a comprehensive AI identity and access management (IAM) strategy that treats every agent as a privileged user by default. While this is one piece of the puzzle, companies also need to understand the broader security posture surrounding their AI Lucys. AI security posture management (AI SPM) gives security teams a continuously updated view of the AI tools, models, data pipelines and cloud services in use, and how they connect. That context helps reveal whether an over-permissioned agent, exposed API or poorly secured dataset could become an attack path long before a prompt injection or other compromise turns AI Lucy into an attacker’s most useful employee.
Here are the contours of an effective AI governance strategy:
• Find the AI Lucys in your organization by inventorying all AI tools and agents in use. List what systems and data these tools and agents can access.
• Ensure that every agent is assigned ownership responsibility, and determine who can justify the access it was granted.
• Grant access based on job requirements, making sure the access expires at predefined intervals or after completion of the task. Review this access like you would a privileged employee’s access.
• Focus especially on logins and behavior. Monitor what agents actually do so a broken process is checked as a potential breach.
• Source the right tools by matching them to your business needs; build a custom AI tool only if you can address all associated risks.
AI Lucy may already be working freely in your organization. Avoid allowing unfettered access. After all, you wouldn’t give a random employee unlimited access, would you?
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?