Meta is failing to remove explicit self-harm images from Instagram and its algorithm is actively encouraging the formation of self-harm networks, according to a new study by Danish researchers. The study, conducted by Digitalt Ansvar (Digital Accountability), found that not a single piece of self-harm content was removed during a month-long experiment on the platform.
Researchers created a private self-harm network with fake profiles, including those of 13-year-olds, and shared 85 items of self-harm-related content, gradually increasing in severity. Despite Meta's claim that it removes 99% of harmful content proactively using artificial intelligence, none of the images were taken down. The study also found that Instagram's algorithm connected the fake 13-year-old profiles to other members of the self-harm group, suggesting it actively helps such networks expand.
Digitalt Ansvar's own AI tool was able to identify 38% of the self-harm images and 88% of the most severe ones, indicating that Meta has access to effective technology but chooses not to implement it. The organisation said this inadequate moderation likely violates the EU's Digital Services Act, which requires platforms to address systemic risks to physical and mental wellbeing.
Meta responded by stating that content encouraging self-injury is against its policies and that it removed over 12 million pieces of suicide and self-injury content on Instagram in the first half of 2024. It also highlighted the launch of Instagram Teen Accounts, which restrict sensitive content for younger users. However, the study's authors argue that Meta avoids moderating small private groups to maintain engagement, and warned that failing to act could have severe consequences, including an increased risk of suicide.
