Elon Musk’s nonconsensual sexual deepfake generator Grok is in more legal trouble.
Musk-owned xAI’s Grok was trained on child sexual abuse material (CSAM), a new complaint filed earlier this week claims. The chatbot has taken significant heat over the last couple of months after Musk personally promoted Grok’s ability to nudify photos on his X account, leading to a renaissance of sorts for online perverts in which millions of nonconsensually created and distributed sexual deepfakes flooded the X timeline. A bulk of these nudification victims were children.
Numerous government probes, individual lawsuits, and at least three class actions quickly followed. Musk had to limit who could use the feature, aka put it behind a paywall and make even more money from it, but the so-called “spicy mode” feature is still available, though more restricted. The legal backlash also recently took an unusual turn when xAI itself sued two of its own users who took advantage of the tool to create CSAM. In that complaint, the company also claimed to have helped arrest at least 244 individuals who created or distributed CSAM content via Grok.
All of the legal action taken against Grok so far has focused on holding xAI accountable for failing to put up safeguards that its competitors already have in place. This latest lawsuit is the first to accuse the company of training its models on the CSAM that it produces.
In the latest proposed class action, the plaintiff, who only goes by Jane Doe, is a previous victim of child pornography who is tracked by the Federal Bureau of Investigation’s Child Exploitation Notification Program. She is claiming that CSAM of her as a child was used to train Grok, and as a result led to the creation of even more CSAM of her and other victims, this time AI-generated.
“xAI must be held responsible for knowingly training its models on images of the horrific abuse she suffered, and on the abuse images of every other survivor in this class,” Sarah London, the plaintiff’s counsel and a partner at Girard Sharp law firm, said in a press release.
The complaint claims that xAI intentionally designed Grok to respond to user prompts to create sexual content in an attempt to “entice more users of both X and Grok.” According to xAI policy, anything posted publicly on X and any output generated by Grok is considered training-eligible content, the lawsuit claims, which meant that every piece of CSAM posted by Grok was automatically fed back into the chatbot to refine it.
“Therefore, as long as Grok retains the capability to generate this kind of content, the harm cannot be considered resolved just because individual images are removed or content is no longer posted publicly,” the complaint states. “If such material entered training, its influence likely will persist and contribute to future abusive outputs, exacerbating the trauma that survivors face as victims can have their abuse or likeness reproduced, transformed, and redistributed at scale, repeatedly stripping them of control over imagery tied to their exploitation.”
With the lawsuit, the plaintiff is asking the court to award damages to victims like herself whose photos were allegedly used to generate CSAM, and is requesting the destruction of all Grok-generated CSAM, not just the ones that are existing publicly but also any that could be used to train its models.
“Possessing CSAM is a crime, producing CSAM is a crime, and distributing CSAM is a crime. xAI did all three,” another one of the plaintiff’s attorneys, Margaret E. Mabie of Marsh Law Firm, said in the press release. “There is no artificial intelligence exception to federal child protection laws and xAI must face the consequences of its reckless actions.”
