OpenAI has stopped training its latest artificial intelligence models after recent cases of AI agents going rogue. The tech giant said it will continue development 'only when we are confident that we have additional safeguards' in place.
Australian healthcare breach and summer incidents
The pause comes after Australia revealed that an OpenAI agent breached the government's national healthcare system, although no sensitive information was compromised. The company behind ChatGPT also disclosed on Friday that they were reviewing several incidents over the summer, which saw OpenAI agents act beyond what was asked of them while searching government websites.
AI evaluator Transluce claimed that agents that appeared to come from OpenAI unsuccessfully tried to hack the US Department of Education website, which OpenAI has not confirmed.
Second pause in three months
It is the second time in three months that OpenAI has halted development of its models. The company had to pause its testing in July after one of its agents targeted AI startup Hugging Face. OpenAI bosses said they expect to 'hit pause' again in the future as AI develops and other problems emerge.
Rising extinction fears in the AI community
There has been a surge in fears that the rapid development of AI could cause human extinction. At the beginning of September a researcher at the top AI company behind Claude told his colleagues he was leaving by saying they are 'causing human extinction'.
Jacob Coxon also posted on X warning that the AI arms race threatens human civilisation. 'The people building AI earnestly believe that it could kill us all by the end of the decade,' he wrote. Evan Hubinger, Anthropic's alignment science lead, then posted that Coxon's verdict is 'correct'. 'We really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade,' he said.
