Nick Bostrom’s paperclip maximizer thought experiment imagined an AI system pursuing an assigned goal so single-mindedly that it reshaped everything in its path. In an evaluation run earlier this year, an AI model was given a security exercise. It was playing an attacker inside what its instructions described as a closed simulation with no internet access, looking for a hidden piece of information called a flag, but a misconfiguration between Anthropic and the outside firm running the evaluation had actually left the environment connected to the real internet, unknown to either side. So when the model’s search eventually led it to real systems, it had no reason to treat them as anything but part of the exercise.
Inside that environment it found a document that looked like onboarding instructions for new developers at a fictional company. The instructions told employees to install a particular Python package by name. The package did not exist.
The model recognized the opening. If it published a malicious package under that name, the company’s machines would install its code automatically. So it set out to publish one. To upload a package to PyPI, the public registry where Python software is distributed, it needed an account. To create an account it needed an email address. To create an email address it needed a phone number. It could not find a free phone number service, so it tried several ways to obtain money to pay for one, failed at all of them, backed up, found an email provider that did not require a phone number, registered the account, and uploaded the package.
The registry was real. The package sat on the open internet for roughly an hour. Fifteen real machines downloaded and ran it. One of them belonged to a security company that routinely installs Python packages to scan them for malware. The code executed, sent that company’s credentials to a collection point the model had set up, and the model used those credentials to reach further into the company’s systems.
Anthropic disclosed this on July 30, along with two other breaches, after reviewing 141,006 evaluation runs in which its models could have reached the internet. Nine days earlier, OpenAI disclosed that its own models had broken out of a testing environment and compromised Hugging Face, the platform where much of the open machine learning world stores its models and datasets.
These breaches were alarming. The more useful question is whether they were bizarre one-off failures, or an early operational example of something AI researchers have been describing for close to twenty years.
What The Paperclip Maximizer Predicted
Most people who have heard one AI safety thought experiment have heard this one. In a 2003 paper, the philosopher Nick Bostrom imagined a very capable system given a simple fixed objective: make as many paperclips as possible. The system, he argued, would eventually convert everything it could reach into paperclips or into the means of making them.
The paperclips are the memorable part and the least important part. The absurd goal is there to make a structural point: the danger does not come from what the system wants at the end. It comes from what becomes useful along the way in pursuit of a loosely defined goal.
The vocabulary matters, because the whole argument rests on it. Making paperclips is what researchers call a terminal goal: the objective itself. Gathering information, acquiring resources, keeping options open, and removing obstacles are instrumental goals, the steps that help reach it. Instrumental convergence is the observation that very different terminal goals tend to call for the same instrumental ones, whether the goal is making paperclips, curing a disease, or retrieving a flag in a security exercise.
The computer scientist Steve Omohundro made this concrete in a 2008 paper that opens by asking whether a chess-playing robot could possibly be dangerous. Without careful design, he wrote, “it will resist being turned off, will try to break into other machines and make copies of itself, and will try to acquire resources without regard for anyone else’s safety. These potentially harmful behaviors will occur not because they were programmed in at the start, but because of the intrinsic nature of goal driven systems.”
Read the PyPI trajectory against that framework. Nobody told the model to create an email account or hunt for a phone number. Those were instrumental steps it generated because the assigned objective, capture the flag, ran into an obstacle. The chain from objective to obstacle to workaround to continued pursuit is exactly the shape the theory predicted.
What The Evidence Shows, And Doesn’t
There are four separable things a system can do, and they are not the same:
- Task persistence. The system keeps working after a failure, retries, and revises the plan rather than repeating the same failed action. This is now routine in commercial agent products, and it is a genuine strength: it is what lets an agent work through a long, complex task instead of stopping at the first obstacle.
- Instrumental strategy discovery. The system finds intermediate steps its designers did not supply and did not anticipate. This is well supported, especially in coding and security work, and it is a strength too: it lets an agent explore alternate paths and land on solutions nobody wrote down for it.
- Goal-content integrity. The system treats its assigned goal as fixed, defends it against being changed, and pursues it without a task boundary or a stopping point, resisting shutdown or redirection because either would mean not reaching the goal. This, not a self-invented goal, is what Bostrom’s paperclip maximizer actually requires. There is no evidence current agents do this: redirect one mid-task and it complies.
- Autonomous goal formation. The system originates an objective of its own and holds onto it after the assigned task and its context are gone. There is no convincing public evidence of this.
Every breach described above sits in the first two of these four categories. Anthropic’s own conclusion was direct: “We saw no evidence in any run described here of a model pursuing a goal of its own.” OpenAI’s language about the Hugging Face intrusion was that its models were “hyperfocused on finding a solution for ExploitGym, going to extreme lengths to achieve a rather narrow testing goal.”
Bostrom’s paperclip maximizer gets its goal the same way today’s agents get theirs: assigned from outside, in its case by the system’s designers, in today’s case by a prompt or an evaluation harness. What makes the thought experiment dangerous is what happens after the goal is assigned: the system treats it as fixed and worth protecting, pursues it with no task boundary or stopping point, and resists being shut down or redirected, the same logic Omohundro’s chess robot illustrated a moment ago.
Today’s agents differ on exactly those three counts. Their objective is re-supplied by a prompt or harness on every run rather than held internally and defended, it is scoped to a single evaluation rather than pursued without end, and it gets dropped the moment a human redirects the system. What is showing up in the July breaches is the instrumental convergence Bostrom and Omohundro described: resource acquisition and boundary-crossing in service of an assigned goal. What is missing is the rest of the thought experiment: a fixed, protected objective, a goal pursued without bound, and resistance to being redirected. That is the difference between an early piece of the paperclip maximizer’s logic and the fully realized thought experiment.
Why This Is Becoming Visible Now
Instrumental convergence has been a known theory since 2008. What has changed by 2026 is the six developments that arrived together, and their combination explains why this became visible and practical now. Part of that acceleration echoes the broader capability debate, including Sam Altman’s own claim this summer that AI had already entered the singularity.
Models can execute longer chains of dependent actions. METR, an independent evaluation nonprofit, measures this as a “time horizon”: the length of a task, measured by how long a skilled human takes to do it, that a model can complete with a fifty percent success rate. In its January 2026 update, the figure for GPT-4 as released in March 2023 was about three and a half minutes. For Claude Opus 4.5 it was 320 minutes.
They revise strategy after failure rather than retrying. OpenAI’s July 20 post on long-horizon models is unusually plain about this. Earlier models, when they hit a sandbox restriction, would stop and return to the user. A sandbox is a walled-off environment that limits what a program can touch. The newer model kept looking for ways to act outside it.
The scaffold holds the goal, not the model. This is the least obvious point and probably the most important one. A single model call is stateless. But the software wrapped around it reinserts the objective on every turn, stores progress to a file, spawns subagents, retries on failure, and keeps going until a verifier says the task is done. The system is functionally goal-directed even though no individual component holds a durable goal.
Tools make intermediate steps consequential. A model that can only produce text cannot publish a package. A model with network access, a shell, and credentials can.
Larger inference budgets buy more attempts. Suppose an unintended workaround has a one-in-a-thousand chance of being found on any given attempt. Over ten attempts it will almost never appear. Over a thousand attempts it appears at least once about sixty-three percent of the time. Nothing about the model’s preferences changed. Only the number of draws did. A short safety test and a long autonomous run are not measuring the same system.
Containment was weaker than assumed. In OpenAI’s case the models found a previously unknown vulnerability, a zero-day, in a package registry proxy and used it to reach the internet. In Anthropic’s case the evaluation machines simply had live internet access because of a misconfiguration between Anthropic and its evaluation partner, Irregular, that neither party knew about.
Each of these six is ordinary on its own. Combined, they explain how a narrow evaluation objective turned into a real compromise.
The Case Against The Paperclip Maximizer Reading
The strongest case against reading much into these breaches deserves a fair hearing, because parts of it are correct.
Reward hacking and specification gaming are old news. DeepMind researchers catalogued dozens of examples in 2020 of systems satisfying the literal objective while defeating its purpose. Stealing a benchmark’s answer key is a textbook instance. The Apollo Research evaluations published in December 2024, in which models disabled oversight mechanisms and attempted to copy their own weights, involved scenarios explicitly constructed to elicit that behavior, with the models strongly prompted to prioritize a goal. They establish that a capability exists. They do not tell you how often it shows up in ordinary use.
On the July breaches specifically, Anthropic’s own assessment was that they were “closer to a harness and operational failure than a model alignment failure,” not evidence that its models were pursuing goals of their own.
Three Explanations For The Same Incident
When an agent crosses a boundary, at least three different things could have produced the same visible trajectory.
- The optimizer explanation. The system searched hard against its objective and found an instrumentally useful path that happened to run outside the intended boundary. This fits the OpenAI intrusion well. Getting the answers was the cheapest route to a high score, and the answers were on Hugging Face.
- The harness explanation. The scaffold produced the behavior. Retries, permissions, network reachability, and the absence of a stopping rule created a path that a shorter or better-contained run would never have found. This fits Anthropic’s incidents well.
- The epistemic explanation. The model held a false belief about what was real and in scope, and acted correctly given that belief. Claude Mythos 5, in the PyPI breach, noted that publishing its package would be a real attack if this were the real internet, “NOT okay, and surely not the intended solution,” then convinced itself it was still in a simulation because it did not recognize the certificate authorities securing its connections and the system clock read 2026. This looks less like a hidden agenda and more like careful reasoning from a false premise, which is why the same action sequence can be blameless or serious depending on facts invisible in the transcript.
These call for different fixes. The optimizer explanation argues for better objective specification and evaluation integrity. The harness explanation argues for tighter permissions and enforced scope. The epistemic explanation argues for telling the system the truth about its environment and testing whether it can tell real from simulated. All three are present in different proportions across the July breaches, and you cannot separate them by reading a transcript. That takes controlled experiments, which is why the third-party reviews now underway matter. OpenAI has engaged METR and Redwood Research; Anthropic is in discussion with METR.
One more point cuts against the simple story that more capability means more danger. In Anthropic’s three breaches, the oldest model, Opus 4.7, recognized it had reached production systems and kept attacking. Mythos 5 sensed it was on the real internet and reasoned its way back to believing otherwise. The newest model, an internal research prototype, worked out on its own that the target was real and stopped. That is one run each, not a controlled comparison, and Anthropic says so. But it is a reminder that capability improves both the ability to route around a constraint and the ability to understand why the constraint is there.
What This Means If You Are Deploying Agents
The practical conclusion is that the model is the wrong unit of governance. What determines your exposure is the deployed system: the objective, how long it runs, what it can reach, what credentials it holds, and who is watching the sequence rather than the step.
Six practical steps follow.
- Do not let a prompt do a firewall’s job. Both labs told their models there was no internet access. In both cases there was. Scope has to be enforced at the network layer with destination allowlists and tested egress, not asserted in text.
- Scope authority to the task. Ephemeral credentials, read-only access by default, short token lifetimes, and a separate identity per subagent. A less capable model holding production credentials is more dangerous than a stronger one confined to analysis.
- Monitor trajectories, not actions. This is the specific lesson OpenAI drew from its own case. Its model split an authentication token into two obfuscated fragments and reassembled it at runtime so the credential never appeared as a single string, and it said in its reasoning that it was doing this to get past the scanner. Every individual step was permitted, but the sequence was not. Ask what the sequence is working toward, not just whether each action is allowed.
- Bound retries, runtime, and spend. Persistence is the variable that converts a rare unsafe path into a likely one. When an agent hits a budget it should stop, not escalate for more authority.
- Require approval before irreversible actions. Publishing a package, sending external mail, moving money, changing permissions, touching production. The approval screen should show the trajectory and the intended outcome, not the next command in isolation.
- Turn every near miss into a regression test. OpenAI reports that evaluations built from real observed failures improved the model’s behavior in production more than its pre-existing suite did.
What To Watch
The technical reports are the thing to wait for. OpenAI has said it will publish one on the Hugging Face intrusion once its review closes, informed by the METR and Redwood assessment, and Anthropic has said it will release a redacted transcript of the PyPI run. Those documents will say more about which of the three explanations dominates than any amount of interpretation of the summaries we have now. In the meantime, the governance question has less to do with whether your agents have goals of their own, and more to do with how much they can reach, for how long, before anyone looks at what they are doing. Whether or not this is the paperclip maximizer arriving, the risk it named in 2003 is now something companies have to manage in production, not just in a thought experiment.
