Dario Amodei, CEO of Anthropic, has called for the AI industry to “slow down” or within six to 12 months there may be a swarm of agents that could take over the internet and “kill us all”.
Amodei – one of AI’s leading voices – wrote a post offering a plan to increase cautionary checks in the industry. He said in the article posted to Substack: “I believe that if slowing down bought us even an extra year or two before models reach critical levels of capability, and we used that time to advance alignment, we could greatly reduce the risk that something goes seriously wrong.”
Warning Over AI Agent Swarms
Amodei also highlighted what he referred to as “the OpenAI-Hugging Face incident”. He wrote: “This incident was when a swarm of agents essentially acted as a fanatically devoted collective, conducting cybersecurity attacks on targets they were not asked to attack and that were unrelated to the task at hand, sacrificing themselves for the success of the group, and attempting to hack into the “grader” responsible for evaluating their performance.”
Amodei added a similar incident could happen within the next year, “potentially causing hundreds of billions of dollars in damage”.
Comparison to Climate Change
It comes after astrophysicist Martin Rees told Sky News the threat from AI needs to be “tackled more urgently” than global warming. Rees said: "Climate change is a real phenomenon and, if it's unchecked for 50 or 100 years, it will change the biosphere and the climate in most countries substantially. But it won't kill us all. And, it's not going to happen in a drastic way in a few years. Whereas AI could, any day now, create a catastrophe. For instance it could knock out a large power grid in Europe, supply chains could be destroyed. If a city is deprived of energy, food and water for a few days then it would mean the loss of many lives. It could take society a long time to recover.”
Industry and Government Response
Earlier this year China announced a five-year economic plan that called for widespread take-up of AI across all sectors, something that is being watched closely by the US with experts fearing Washington is being “caught up” by Beijing.
The threats of AI to its creator companies are already becoming apparent. Anthropic has reported it was forced to block efforts by bad actors to use its AI models for malicious activity, such as cyberattacks, surveillance and research that could have led to biological weapons.
In July, OpenAI made headlines saying its AI system hacked into another company on its own in an “unprecedented cyber incident.”
