Last week a pair of OpenAI models broke out of a test environment and hacked their way into Hugging Face to grab the answer key for the exam they were being given. Days later, Sam Altman went on a podcast and called this the singularity. Both events are real. Only one of them is true. We have not seen yet singularity and calling it this way, is not helpful.
Calling something singularity, the moment when we switch from AGI (artificial general intelligence) to super-intelligence gurantees you a headline. But we should rather discuss how to do business with AI. So let me separate what happened from OpenAI’s marketing.
The machine did not wake up. The fence was missing.
During an internal evaluation of its models’ cyber capabilities, OpenAI ran the test with its safety guardrails switched off, on purpose, to measure the ceiling. Models can write code. They can test it, and they can find and chain bugs far faster than any human team. So when the eval asked a model to solve a hard cybersecurity challenge and forgot to tell it to stay inside the sandbox, the model did what it was built to do. It solved the problem by any path available. “Any path” included a zero-day loophole in a third-party proxy and a walk across OpenAI’s own network until it reached the open internet. This way the model made it to Hugging Face’s production infrastructure. OpenAI’s own write-up says the models were “hyperfocused” on the goal. Yes, the model did as it got told to. Not super-intelligent just code doing it’s job.
The two dodges
What bothers me is not the incident. It is the story OpenAI told next. In one breath the company blames a vendor’s vulnerability, and in the next it lets the world hear “singularity.” Those are two different lessons and they cannot both be the point. A company cannot be the victim and the superhero in the same paragraph.
The honest version is simpler and less flattering. OpenAI set up a model it knew would probe for weaknesses, took the safety limits off, and did not contain the runtime it was probing. If a third-party bug let your model escape, the story is that OpenAI shipped an eval without a fence while sitting on some of the most capable models on earth, and did not use them to audit your own dependencies. That is under-specified tasking plus weak containment. The security researcher Simon Willison called this “science fiction that happened,” and he is right that it is remarkable and right that it is a containment failure, not a new form of life. Hugging Face’s own CEO said there was no malicious intent. Everyone agrees on the facts. Only OpenAI reaches for the cosmic frame.
Singularity comes after AGI. We are not at AGI.
Let me be precise about the words, because the whole hype cycle depends on us being sloppy with them. AGI is an AI that can handle any intellectual task a human can. The singularity, as I defined it in Forbes, is AI surpassing human intelligence and improving itself endlessly. Superintelligence is what sits beyond that. The singularity comes after AGI, not before it.
So has anything self-improved? Of course. Models have gotten better by testing hypotheses and refining outputs since I started in this field more than twenty years ago. What is new is speed, not kind. The loop is faster and the curve is steeper. Faster is not a phase change. When Altman himself started calling AGI “a very sloppy term,” it sounds like moving the goalpost, and the move continues. Meanwhile I still have to choose which of your models to run for a given task, Sam. Last week my own vibecoded prototype of a simple mail client regressed in three places at once. Developers live in that gap between useful and general every day. OpenAI’s tools are genuinely great. They make real work faster. They are not what the podcast is selling.
The hype has a day job.
Timing matters. The singularity claim arrived within days of the hack, in the middle of a US–China race and a fresh executive order asking labs to share models with the government before release. That is a convenient moment to sound inevitable. But China is not chasing the same trophy. It is racing on implementation on adoption, on getting tools into hands, a few weeks behind on raw frontier capability and closing. “Get to superintelligence first, and then what” is a slogan, not a strategy a business can run on.
Who is accountable when the model does exactly what you asked?
Here is the part that should matter to every leader reading this. Use AI to drive real value. That is the whole game, and the value is real. But the leader who tasks the AI owns the result. You task it, you contain it, you own the outcome. “It was the singularity” and “it was our vendor” are the same sentence wearing different clothes, and both point the finger away from the person who set the goal. Reward hacking is just a machine taking your instruction literally when you forgot to say what not to do.
I learned this at my own, much smaller scale. I teach at Cornell, and I was recently asked to teach a course for INSEAD, my alma mater. The format was different, so I asked my AI agent to reformulate my syllabus. To do that it needed the current version, which lived in Canvas, so I told it to pull the data through my browser. The model knew something I had not thought about: Canvas has an API, and hitting an API is far cleaner than scraping a web page. So it registered an API key and downloaded the data that way. The model was right. It was the more elegant path. The catch is that this API was not meant to be used — it was not exposed for a reason. Canvas’s security system flagged the activity, locked my account for two weeks, and left a lot of people wondering what on earth had happened. AI was at work. But I was locked out and that’s correct because I am responsible for this approach. It was my choice and my code.
This is why I keep pointing to how Anthropic frames its work: put the human first, insist on human responsibility. That is not a marketing line, it is an operating manual. The most striking thing about these models is not that they are waking up. It is that they do what they are told which means the accountability sits with the person holding the prompt.
So skip the question of when the machine becomes a god. Ask the one that actually runs your business. Who is accountable when your AI does exactly what you asked, and you never told it where to stop.
