Dario Amodei’s Anthropic and Sam Altman’s OpenAI were in talks earlier this year on a deal to stress-test each other’s AI models for potential safety flaws, according to a report.
They began negotiations on a “legally binding deal” before high-profile incidents such as OpenAI’s accidental hack of rival firm Hugging Face occurred, The Information reported Monday, citing a person with direct knowledge of the talks.
Lawyers for the AI giants were drawing up the terms of the deal, which would have involved each company subjecting the other’s new models to a battery of tests aimed at finding flaws or other “hidden dangers,” the report said.
However, it’s unclear whether the agreement was ever finalized.
The idea of implementing some form of peer review among leading AI labs has gained steam in recent days. Elon Musk, who founded Grok-maker xAI, argued that top US labs and their Chinese counterparts should test each other’s models during an appearance at the All-In Summit in Los Angeles last week.
The AI safety debate exploded into the mainstream earlier this month when Jacob Coxon, a 27-year-old former employee of Anthropic and OpenAI, said leading AI companies are “gambling with our lives” by speeding ahead of development without proper guardrails in place.
Just a few days later, Amodei published an essay calling for an industrywide slowdown. The Anthropic boss argued that AI bots could be less than a year away from “taking over the entire internet” and “potentially causing hundreds of billions of dollars in damage” without improved safety standards.
As The Post has reported, Anthropic has faced sharp scrutiny over Amodei’s suggestion to rely on “embedded third-party evaluators” to oversee the safety of advanced AI models. Amodei endorsed METR – a group with extensive ties to controversial Effective Altruism movement and Anthropic’s own investors – for the job.
Critics say such ties are conflicts of interest – and that Anthropic’s close relationship with METR and Redwood Research mean those AI watchdogs aren’t truly independent.
Altman also said he agreed with the concept of embedded third-party AI safety watchdogs at leading labs, though he stopped short of endorsing a particular group to take up the task.
One of the leading skeptics of calls for an industrywide slowdown is President Trump, who has labeled mounting concerns as a “hoax.” On Saturday, the commander-in-chief said he would create an “AI Force” and name a new AI “czar” to help oversee the industry.
Representatives for OpenAI and Anthropic did not immediately return a request for comment.