Anthropic has claimed to have blocked "malicious" AI activity that could cause it to develop biological weapons. The California-based company said criminals and state-sponsored entities previously attempted to use Anthropic's intelligence models to create bombs, deadly pathogens and to survey dissidents.
Report details five case studies
In a 154-report Anthropic said on Thursday: "The cases we share here aren’t typical misuse, but rather examples of the most notable and novel threat activity we’ve identified to date. We’re publishing this work because we believe we have a responsibility to disclose malicious misuse of our services."
Anthropic shared five case studies where it said biological researchers circumvented safeguards designed to stop users from "unsupported regions" and attempted to hide the point of their work. "Biological misuse is one of the most serious risks of frontier AI models," the company said. "Without the correct safeguards, such capabilities could have catastrophic consequences."
Accounts banned and timeline
Anthropic said it has since banned the accounts, though it did not share the names of the institutions or nations where the misuse happened. The company said the "malicious use" of Claude Haiku, Sonnet and Opus models had been prevented between December 2025 and August 2026.
The report was released just days after ex-Anthropic employee Jacob Coxon public said he quit the company due to concerns it was not acting responsibly when creating the technology. In a more alarming claim, he said the technology could cause human extinction by the year 2030.
Former employee warns of extinction risk
"Do not underestimate the power of this technology," Coxon said in a September 9 X/Twitter post. "The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible - but I hear the same people express fear privately. No other human activity poses this level of danger."
Several current Anthropic employees have posted public agreements with Coxon. AI experts have highlighted potential threats posed raised in the company's intelligence report.
Industry collaboration urged
Anthropic said the AI industry as a whole needed to collaborate with governments in order to address concerns as well as regulations. It said: "As models become increasingly capable, their risks will increase, unless AI developers and society’s defenders act to make them safer."
Anthropic head of threat intelligence Jacob Klein told the New York Times added it was "an incredibly nuanced situation." He added: "You are not seeing someone in a comic book kind of way say, 'Hey, I want to build a biological weapon to kill everybody.'" The company claimed it had shared its findings and intelligence with authorities as well as industry partners.
