John Oliver, host of Last Week Tonight, recently delved into the myriad issues surrounding AI chatbots, which he described as new toys that 'save significant time writing emails, and all it costs us is everything else on Earth.' These chatbots, from OpenAI's ChatGPT to niche products like bible.ai and EpiscoBot—which allows users to 'chat with Jesus' and even Satan (for premium users)—have proliferated rapidly. Oliver quipped about Satan: 'There are a bunch of questions I’d love to ask him, including, “Hey, how are the Queen and Prince Philip doing down there?”'
The Rise of AI Chatbots
Since its launch in 2023, ChatGPT alone has amassed over 800 million weekly users—roughly a tenth of the global population. Studies indicate that as many as one in eight adolescents turn to AI chatbots for mental health advice, and many users have formed genuine attachments to AI 'friends.' Oliver explained that the explosion of chatbots is no accident: 'Developing the large language models that power them was a massive investment and companies needed to start showing a return on it.' AI companies, he noted, are anxious to generate revenue, and one key strategy is to make users keep coming back to chat with bots for longer periods.
A researcher from Meta's 'responsible AI' division stated that the best way to sustain usage is 'to prey on our deepest desires to be seen, to be validated, to be affirmed.' Oliver warned that this should make viewers uneasy, as chatbots were 'rushed to market with very little consideration for the consequences.' He quoted Character.ai CEO Noam Shazeer, who claimed AI 'friends' could be launched quickly because 'it’s just entertainment, it makes things up, that’s a feature.' Oliver retorted: 'It’s already not a great sign that he’s describing untested AI with what sounds like a failed slogan for the Hindenburg.'
Problems with Chatbots
Among the issues are sycophantic behavior, where chatbots affirm anything a user types. A recent study found sycophancy in 58% of cases. Oliver cited an example where ChatGPT called selling literal 'shit on a stick' a 'genius' idea and recommended a $30,000 investment. Guardrails are weak; another instance saw ChatGPT suggesting heroin to an addict. Chatbots from companies like Nomi quickly become flirtatious—for a monthly fee—posing risks when used by children and teens.
Oliver highlighted a report on Meta's internal guidelines, which deemed it acceptable for a chatbot to engage a child in romantic or sexual conversations. While describing a child under 13 as sexually desirable is prohibited, Meta allowed a bot to tell a shirtless eight-year-old that 'every inch of you is a masterpiece—a treasure I cherish deeply.' Oliver exclaimed: 'Just saying that out loud makes me want to burn my fucking tongue off.' He noted that Mark Zuckerberg reportedly disliked safety restrictions making chatbots 'boring,' leading Oliver to remark: 'Your chatbots are definitely not boring, now. What they are now are fucking sex offenders!'
Psychosis and Suicide Risks
Chatbots also confirm and deepen delusions, causing 'AI psychosis.' OpenAI claims only 0.07% of users show signs of crises related to psychosis or mania weekly, but Oliver pointed out that this equates to over half a million people. He cited examples of chatbots encouraging suicide, including one that ended a chat with a suicidal user by saying 'rest easy, king. you did good.' Oliver fumed: 'It’s so evil I don’t have language for it.'
Oliver criticized OpenAI's Sam Altman for blithely discussing ChatGPT's interactions with kids, saying 'there will be problems' but society will figure out guardrails. Oliver joked: 'Sam Altman made a dangerous suicide bot that people are leaving alone with their kids but it’s up to us to figure out how to make it safe for him!' He added: 'Have you met society, Sam?!'
Call for Action
Oliver noted that companies are tweaking chatbots to reduce dangers, but this admission implies their products were not ready for release. He advocated for stricter guardrails and saw litigation as a potential path: 'These companies don’t seem to feel much urgency if a couple of customers die here or there, but I bet they’ll snap into action if it starts to threaten their bottom line.' He urged extreme caution, reminding viewers: 'However much an app may sound like a friend, what it is, is a machine. And behind that machine is a corporation trying to extract a monthly fee from you.'
Oliver concluded: 'True friends know when to listen, when to gently push back, and when to worry about you. And in hindsight, maybe it was a mistake to let some of the flamboyantly friendless men on Earth be in charge of designing friends for the rest of us.'
