Anthropic says scientists used its Claude AI in sensitive biological research that the company feared could have applications in weapons development — including gain-of-function virus work and experiments involving pathogens with pandemic potential.
The company said it could not determine the researchers’ ultimate intent because the same scientific work can have legitimate medical uses as well as dangerous applications.
Anthropic said it banned associated accounts and disrupted access where it identified violations of its safeguards and policies.
The findings, disclosed Thursday in a sweeping report on misuse of Anthropic’s models, provide real-world context for warnings from former Anthropic researcher Jacob Coxon that increasingly powerful AI systems could eventually pose an extinction-level threat.
One case involved a scientist seeking Claude’s help preparing a grant application for gain-of-function research on chikungunya, a mosquito-borne virus that can cause severe pain and fever.
The proposed work sought to identify mutations affecting transmissibility and immune evasion and then engineer those changes into the virus, according to Anthropic.
The company said the research could have legitimate applications, including vaccine and therapeutic development.
But Anthropic said the case became particularly concerning because the work was intended to take place at a military research institute.
Anthropic said it could not establish whether the research was intended to be weaponized.
The latest revelations come just a day after Coxon warned that increasingly powerful AI systems could eventually pose an extinction-level threat to humanity, telling NBC News there was a “substantial probability that this technology could kill everyone” and insisting, “This isn’t hyperbole. It’s not a marketing stunt.”