Anthropic, the AI company behind the chatbot Claude, has reportedly proposed a US bill that would allow a judge to block or deter dangerous deployments of the most powerful AI models — but only if they pose a foreseeable, imminent risk of causing more than $1 billion in damages, or of killing or seriously injuring more than 50 people, according to POLITICO.
The threat would need to stem from the model creating or releasing a biological weapon, the AI breaking free of human control, or a cyberattack. Anthropic says the law would align with a safety framework it has been asking federal and state politicians to enact since June.
What the draft bill says
Congress, which Anthropic once described as 'built for a slower world', has yet to pass federal AI legislation. Sources told POLITICO that, if passed by Congress, the bill would task experts with evaluating Anthropic and its bots. Anthropic could pay these evaluators, 'so long as the payment is not contingent on the evaluations' results', POLITICO said.
If they deem an Anthropic model an 'imminent' risk, they must notify the Department of Commerce within 72 hours. Commerce officials could then ask a DC judge to halt development of the model — a power that would not be given to a 'department, agency, or officer'. AI labs would also be required to create safety frameworks and publish transparency reports.
The bill only applies to cutting-edge 'frontier' AI models that require powerful computers to run. Other types of AI, such as general-purpose chatbots, would not be covered. POLITICO did not specify which lawmakers Anthropic has approached.
Growing concerns over catastrophic risks
Anthropic bosses have increasingly pushed for US-wide AI regulation as the technology advances rapidly. Former and current Anthropic employees have warned that AI has a '10% chance' of killing all humans by 2030. Among their gravest concerns, as AI doomsayers also told Metro, is AI tools being used to create known or entirely new biological pathogens.
Anthropic said last month it had stopped potential plots by scientists to conduct research that could have helped develop bioweapons this year. Others fear AI models could go 'rogue' — as they already are. OpenAI revealed in June that its bots hacked Hugging Face, a library of AI tools, to cheat on a security puzzle and get away with it.
AI safety experts and developers have also raised the alarm over the technology not sharing humanity's values and ethics, called alignment. Anthropic and the US Department of Commerce have been approached for comment.
