Anthropic said artificial intelligence is advancing so rapidly that an industry-wide pause is needed to slow the pace of development while companies get a handle on potential societal risks.
The AI juggernaut published a blog post Thursday warning that AI systems could soon be able to improve themselves without human intervention, resulting in chaos.
Critics and rivals have cast similar statements in the past from Anthropic — which dethroned OpenAI as the most valuable AI company last month – as part of a bid to hobble the competition in the red-hot race to develop advanced artificial intelligence.
Anthropic has long warned that the power of its models could pose risks to humanity. Anthropic’s Chief Executive Dario Amodei feuded for months with the White House over AI safety related to the use of its systems in military applications.
Anthropic called on top AI labs to weigh slowing the pace of development and offered internal data that showed the speedy improvement of its own models.
“We believe it would be good for the world to have the option to slow or temporarily pause frontier AI development to enable societal structures and alignment research to keep up with the advance of the technology,” the post stated.
It was written by Anthropic’s head of internal research Marina Favaro and head of policy Jack Clark.
The ability to slow global AI development would “likely be a good thing,” they added.
OpenAI CEO Sam Altman previously accused Anthropic of doing “fear-based marketing.”
“It is clearly incredible marketing to say, ‘We have built a bomb, we are about to drop it on your head. We will sell you a bomb shelter for $100 million,’” he said last month.
Anthropic’s post outlined how model advances appear to be moving toward “recursive self-improvement,” or AI systems that can improve on their own without human intervention. AI experts have warned that the phenomenon could wreak havoc on society.
“This would revolutionize knowledge work and government services, but could also be turned to harmful ends, from authoritarian surveillance of whole populations to influence operations that tailor manipulation to each individual and run at a scale no human team could match,” the post stated.
It acknowledged that self-improving AI hasn’t happened yet, and isn’t inevitable.
Anthropic’s valuation reached a whopping $965 billion in its $65 billion Series H funding round last month. The firm had long been seen as an upstart seeking to catch up to OpenAI’s early dominance in the field.
Anthropic’s Thursday post went as far as to propose a global agreement for the slowdown and how to verify adherence to it.
Some have criticized the lab’s many warnings about the danger its own tools could pose as a marketing tactic.
University of Pennsylvania’s Wharton School professor Ethan Mollick posted on X: “There is a bit of navel-gazing, some marketing, and a lot of very sincere beliefs about what Anthropic thinks is likely in the near future of AI.”
Anthropic’s leaders have insisted they’re putting AI safety ahead of growth and want to facilitate more discussion of risks.
One online critic took a political angle on the lab’s posting, writing on X: “They’re trying to pause until a Democrat gets back in the White House.”
