Earlier this week, the California Attorney General’s office announced that xAI is investigating reports that its chatbot startup, Grok, is being used to create a nonconsensual sexual image of women and small children. On Friday, the government followed up by sending a cease-and-desist letter to the company, demanding that it take immediate action to stop the production of nonconsensual intimate images and CSAM – child sexual abuse material.
“Today, I sent xAI a cease-and-desist letter, demanding that the company immediately stop the creation and distribution of deepfake, nonconsensual, intimate images and child sexual abuse material,” said California AG Rob Bonta in a press release. “The creation of this material is illegal. I expect xAI to comply immediately. California has zero tolerance for (CSAM).”
The AG’s office also claimed that xAI appears to “facilitate the large-scale production” of nonconsensual porn, which is “used to harass women and girls on the internet.” The agency said it expects xAI to prove it is taking steps to address the issue within the next five days.
At the heart of the backlash is Grok’s “spicy” mode feature, which xAI created to generate explicit content. The issue has spread beyond California; Japan, Canada, and the United Kingdom have opened investigations into Grok, and Malaysia and Indonesia have provisionally blocked platform totally. Although xAI imposes some limitations on image editing features last wednesdayThe California AG’s office moved ahead with its cease-and-desist letter.
X’s safety account is already there cursed this type of user activity, said: “Anyone who uses or requests Grok to create illegal content will suffer the same consequences as uploading illegal content.” TechCrunch reached out to xAI for comment and was met with an automated email that said “Legacy Media Lies.” TC also reached out to the California AG’s office for more context.
The advent of free generative AI tools has caused disruption swell from non-consensual sexual material. Many platforms have been grappling with this problem, not only X. The sordid activity has attracted the attention not only of state leaders but also of Congress. Indeed, on Thursday, members of parliament sent a letter to executives from several companies—including X, Reddit, Snap, TikTok, Alphabet, and Meta—asking how they plan to prevent the proliferation of sexual deepfakes.