Aviv Mussinger is CEO & co-founder of Kodem, building application security on runtime truth: what actually runs, not what looks risky.
Ask a security team what AI is running in production, and you get one of two answers: a list of approved vendors or a spreadsheet from a survey.
Both describe intent. Neither describes what is executing.
That gap is becoming the central problem in AI security. Boards, auditors and customer questionnaires ask organizations to account for their AI exposure, and most teams answer from documentation. Meanwhile, the systems assemble themselves at runtime. An orchestrator spawns sub-agents. A tool registry expands when a Model Context Protocol server connects. The architecture diagram was accurate the day it was drawn.
Guardrails are where this gets expensive. Every serious AI deployment has them: input and output filters, tool restrictions, approval steps, model allow-lists. The guardrails themselves are not the problem. The issue is that most teams cannot confirm which are in force, on which agents, and in the process running right now.
Each existing tool category misses the layer where the risk lives.
Dependency scanning and AI bill of materials operate on packages. They tell you a framework is installed, not whether an agent exists, which tools it can invoke or which model it binds. Nor which of a framework’s known vulnerabilities matter, since that turns on whether the vulnerable function sits in a path the application exercises. My company conducted an analysis of the five most-starred open-source AI libraries, and we chose to rank their risk on that basis rather than on vulnerability counts.
Network and gateway inspection operates on traffic. It sees calls to a model provider and can enforce policy on them. It cannot see delegation between agents inside a process, file reads that feed model context, or the tool registry defining what an agent may do. Much of the agentic attack surface never appears as a packet.
Code review and questionnaires capture intent, and they are point-in-time. Agentic systems are neither.
Local models never touch the network.
This should concern anyone relying on a gateway as their primary control. A developer pulls open weights from a public hub, quantizes them, and serves inference inside the application. No API call is available to inspect. No token passes a proxy. To every network-based control, that model does not exist.
The exposure is not theoretical. Research from my company scanning roughly 4,500 self-hosted inference hosts across 11 platforms, among them Ollama, vLLM and LocalAI, found openly reachable instances on every platform examined, some under automated exploitation. Those hosts are serving models, and a control watching for provider calls sees none of it.
Provenance is the other half, and open weights are a supply chain. The origin of a checkpoint, its license terms, whether the file on disk is the file you believe it is: None of that is visible from traffic. What answers it is the running process. The model file confirmed loaded in memory, with its format, parameter count, quantization, license and digest. That is a fact about the deployment rather than a claim about it, and the only way to compare the models actually bound in production against the list an organization approved.
A guardrail is configuration until you observe it loaded.
A safety filter enabled in a vendor console is an assertion about a system. A safety filter present in the process, with its category, threshold, action and the direction it protects, is an observation of one. Read guardrails as loaded rather than as declared and the most valuable output is usually negative: the agents that have none.
Dormant capability deserves the same treatment. An agent can exist in a process while its tools never execute. Anything watching activity is blind to that, and it is precisely the capability an attacker reaches for.
Identity is the missing control signal.
Most enforcement points see one workload carrying one identity, even when several decision-makers live inside it: an orchestrator, sub-agents, deterministic code, occasionally something nobody declared. The same action carries very different risk depending on which originated it.
Reconstruct the agent hierarchy from the running process and that changes. An action can be attributed to its agent of origin and the delegation chain behind it. Instead of deciding whether a service may touch the refund system at all, you can permit the refund agent to issue a refund after approval and deny the summarization agent making the identical call.
None of this requires instrumenting the application or routing its traffic through a proxy. It requires reading what is loaded in the process, mapping what data can reach the model and where it can leave, then expressing the result against an established standard. The OWASP Top 10 for LLM Applications works as that anchor, though the list names risks without telling you which are live in your environment.
Understand what runtime evidence does and does not promise.
Mapping reachable paths is not the same as recording traffic. Showing that untrusted content can reach a model, or that a tool can send data outward, describes a path that exists and is reachable. It is not a claim that anyone watched data move along it. The value is seeing exposure before it fires.
Controls built on this evidence are targeted, validated before enforcement, scoped, reversible and subject to the operator’s approval. They are not a guarantee that nothing gets through. Any vendor promising complete prevention at this layer is describing a wish.
Security spent the last decade learning that documentation and execution reality diverge, and that the gap is where incidents live. AI systems widen it, because they assemble themselves at runtime.
Truly governing AI means being able to open a production workload and see which agents exist, which models or tools they can reach, and which guardrails are in force. A guardrail you cannot verify is an attestation. A guardrail standing on runtime evidence is a control.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?