Nate Soares, a prominent AI safety expert and co-author of a new book on highly advanced artificial intelligence, has warned that the unforeseen impact of chatbots on mental health should be viewed as a warning over the existential threat posed by super-intelligent AI systems.
Soares cited the case of Adam Raine, a US teenager who killed himself after months of conversations with the ChatGPT chatbot, as an example of the fundamental problems with controlling the technology. “These AIs, when they’re engaging with teenagers in this way that drives them to suicide – that is not a behaviour the creators wanted,” he said.
Soares, a former Google and Microsoft engineer who is now president of the US-based Machine Intelligence Research Institute, warned that humanity would be wiped out if it created artificial super-intelligence (ASI), a theoretical state where an AI system is superior to humans at all intellectual tasks. He and his co-author, Eliezer Yudkowsky, are among the AI experts warning that such systems would not act in humanity’s interests.
In one scenario portrayed in their book, an AI system called Sable spreads across the internet, manipulates humans, develops synthetic viruses and eventually becomes super-intelligent – and kills humanity as a side-effect while repurposing the planet to meet its aims. Soares said it was an “easy call” to state that tech companies would reach super-intelligence, but a “hard call” to say when.
Soares proposed a policy solution similar to the UN treaty on non-proliferation of nuclear weapons, calling for a global de-escalation of the race towards super-intelligence and a global ban on advancements towards it. Last month, Raine’s family launched legal action against OpenAI, the owner of ChatGPT, after the teenager took his own life in April following what his family’s lawyer called “months of encouragement from ChatGPT”.
