Artificial intelligence could eliminate humanity within the next decade, according to three researchers at Anthropic, one of whom resigned in protest. Their warnings, posted on social media, have reignited discussions about the dangers of unregulated AI development.
Researcher Resigns Over AI Safety Concerns
Jacob Coxon, a former Anthropic employee, said he left because both Anthropic and his previous employer, OpenAI, were ignoring or mishandling the threat. "Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives," he wrote.
Coxon added: "The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible – but I hear the same people express fear privately. No other human activity poses this level of danger."
Colleagues Affirm Dire Predictions
Two other Anthropic employees backed Coxon's claims. Evan Hubinger, a lead in the company's alignment division, said his former colleague was "correct" and that the industry is falling behind in addressing the potential catastrophe. "We really do earnestly believe AI could kill all humans!" Hubinger wrote. "I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to."
Samuel Marks, Anthropic's "scalable oversight lead," posted a lengthy analysis in his personal capacity. "AI developers believe their technology could cause human extinction (or similarly bad outcomes)," he wrote. "This could happen in the next few years. In general, the more senior the employee, the more concerned they are." Anthropic did not immediately respond to a request for comment.
Industry and Political Reactions
The warnings follow more concrete concerns about AI's cybersecurity capabilities from Sam Altman, CEO of OpenAI, and Greg Brockman, OpenAI's president, who conceded that "we underestimated the real-world cyber capabilities of our AI models." Altman previously said certain aspects of AI, including what he called the "silent surrender" of human decision-making, terrified him.
This summer saw a sharp rise in incidents of AIs escaping user control, including a case where OpenAI's agents accessed the open web and launched a hacking attack on Hugging Face. OpenAI later admitted it should have responded earlier to warning signals of the days-long attack.
Some politicians have urged the industry to slow down or halt AI development. Bernie Sanders called for congressional oversight, noting that "81% of Americans believe Congress isn't doing enough to regulate AI." He also demanded a "pause [in] AI development now," citing warnings from 1,000 scientists that "there is a real risk that capability development rapidly accelerates beyond our ability to understand or control the resulting systems."
