Social media companies have always struggled to moderate user content, and AI is only making the problem worse.
A new investigation found that Facebook allowed accounts sharing disturbing AI-generated videos of coaches inappropriately touching young athletes to remain online, even after users reported them. The accounts were only removed after journalists started asking questions.
Futurism reporter Maggie Harrison Dupré reported Monday that Facebook was hosting numerous accounts dedicated to sharing AI-generated videos of older male coaches inappropriately touching young women and girls during gymnastics, ballet, and yoga practice.
The disturbing videos show coaches running their hands over athletes’ backs and inner thighs. Some even depict coaches touching the young athletes’ genitals over their clothing or kissing them on the mouth. Other clips try to pass off the inappropriate behavior as coaches simply trying to improve the athletes’ technique.
“I do consider this abuse material. These images sexualize and normalize abusive interactions between coaches and young athletes within a sporting environment where significant power imbalances already exist,” Kim Shore, a gymnast and advocate for athlete safety who advises Gymnasts for Change International, told Futurism.
To see how Facebook would respond, Futurism used a regular account to report 10 pages sharing these videos.
A week later, Facebook said eight of the pages did not violate its rules and would not be removed. The other two accounts had disappeared, but it was unclear whether that was because of the reports.
“We use a combination of technology and human reviewers to process reports and identify content that goes against our Community Standards. To keep our review process as fair as possible, we use the same set of Community Standards to review all reports,” the company said in eight identical notifications about the reported accounts. “We’ve taken a look and found that the page doesn’t go against our Community Standards.”
But Facebook’s response changed after Futurism contacted Meta’s media relations team.
According to Dupré, the eight accounts were removed shortly after the news outlet reached out to Meta. Several of the earlier notifications were also updated to say that the pages did, in fact, violate Facebook’s rules and had been taken down.
Meta confirmed the removals in a statement to Futurism, saying that content depicting abuse or exploitation has no place on its platforms, whether it’s real or AI-generated.
When Futurism asked Meta why it had responded differently to reports from a regular user than to questions from journalists, the company defended its content moderation efforts.
“While there are times we miss things, we work hard to enforce our rules, which is shown by the fact that just last quarter, we proactively found and removed 98 percent of child endangerment posts before they were ever reported to us,” the company responded.
Still, that doesn’t explain why Facebook initially cleared the accounts.
Meanwhile, Meta has also been testing AI to improve content moderation. In its latest Community Standards Enforcement Report, the company said initial tests found that large language models (LLMs) “make 13% fewer mistakes than humans when enforcing against violating content while finding 10% more actual violations.”
But whether AI can actually help improve content moderation remains to be seen. For now, the tech appears to be creating plenty of new problems, as Meta isn’t the only social media platform dealing with a profileation of disturbing AI-generated content.
Earlier this year, Elon Musk’s social media site X faced backlash and sparked multiple government investigations after some users began using its AI chatbot Grok to generate sexualized images of people without their consent, including images involving minors.
The incidents raise questions about whether social media companies can keep up with the harmful content their own AI tools are helping create.
