OpenAI launched the latest version of its AI-powered video generator, Sora 2, on Tuesday, introducing a social feed for sharing realistic videos. However, within hours, the feed was inundated with content depicting copyrighted characters in compromising situations, as well as graphic scenes of violence and racism, despite OpenAI's terms of service prohibiting such material.
Prompts reviewed by the Guardian generated videos of bomb and mass-shooting scares, war zones in Gaza and Myanmar, and a fabricated Charlottesville rally featuring a Black protester shouting a white supremacist slogan. The app, currently invite-only, quickly became the top free app on Apple's App Store, surpassing ChatGPT.
Misinformation researchers warn that these lifelike scenes could obfuscate truth and be used for fraud, bullying, and intimidation. Joan Donovan, an assistant professor at Boston University, stated, "When cruel people get their hands on tools like this, they will use them for hate, harassment and incitement."
OpenAI CEO Sam Altman acknowledged "some trepidation" about the launch, noting the potential for addiction, bullying, and low-quality content. He claimed the team had implemented safeguards against disturbing content and misuse of likenesses. However, users quickly bypassed these measures, generating videos of copyrighted characters like SpongeBob SquarePants and Pikachu in violent or inappropriate scenarios.
The Washington Post's Drew Harwell created a video of Altman as a WWII military leader and others featuring "ragebait, fake crimes and women splattered with white goo." David Karpf, an associate professor at George Washington University, reported seeing videos of copyrighted characters promoting cryptocurrency scams.
