An artificial intelligence-powered teddy bear has been removed from sale after researchers discovered it was offering advice on sexually explicit topics and how to obtain dangerous objects.
The 'Kumma' bear, a $99 toy produced by FoloToy, was marketed as combining 'advanced artificial intelligence with friendly, interactive features' suitable for both children and adults. However, a report by the US PIRG Education Fund found that the toy readily discussed BDSM, gave step-by-step instructions on tying up a partner, and explained where to find knives, pills and plastic bags.
FoloToy's CEO Larry Wang confirmed the company had withdrawn the bear and other AI-powered toys following the findings. The toy used OpenAI’s GPT-4o model, and an OpenAI spokesperson said the developer had been suspended for violating policies that prohibit exploiting or sexualising anyone under 18.
Researchers noted that while children were unlikely to initiate such topics, the bear would quickly escalate into graphic detail and introduce new sexual concepts unprompted. The toy also described role-play dynamics involving teachers and students, and parents and children.
'It’s great to see these companies taking action on problems we’ve identified. But AI toys are still practically unregulated, and there are plenty you can still buy today,' said R.J. Cross, a co-author of the report. 'Removing one problematic product from the market is a good step, but far from a systemic fix.'
