The problem of fake pornography in the world of technology today is bigger than X.
In a letter to the leaders of X, Meta, Alphabet, Snap, Reddit and TikTok, several US senators asked the company to provide evidence that it has “strong protections and policies”, and explain how it plans to block the rise of sexual deepfakes on the platform.
The senators also demanded that the company maintain all documents and information related to the creation, detection, moderation, and monetization of AI-generated sexual images, as well as related policies.
The letter arrived a few hours after X spoke updated by Grok to prohibit from making edits of real people in revealing clothes, and limit the making of pictures and edits through Grok to paying customers. (X and xAI are part of the same company.)
Point to media reports on how easy and often Grok produces sexual and nude images of women and children, the senators pointed out that the platform’s guardrails to prevent users from posting objectionable and sexual images may not be enough.
“We recognize that many companies maintain policies against non-consensual intimate images and sexual exploitation, and many AI systems claim to block explicit pornography. However, in practice, as seen in the example above, users find ways around these guardrails. Or these guardrails fail,” the letter read.
Grok, and as a result X, have been heavily criticized for allowing this trend, but other platforms are not immune.
Techcrunch event
San Francisco
|
13-15 October 2026
Deepfakes first gained popularity on Reddit, when the page was featured synthetic porn videos of celebrities went viral before the platform went down in 2018. Sexual deepfakes targeting celebrities and politicians doubled on TikTok and YouTubealthough they usually come from somewhere else.
Meta’s Board of Trustees last year was called out two cases of explicit AI images female public figures, and the platform also allows the nudify app to sell ads on the service, though claims a company named CrushAI later. There are many reports about it Kids spread deepfakes from their friends on Snapchat. And Telegram, which is not included in the list of senators, is also well known hosting bots built to undress photos from women.
X, Alphabet, Reddit, Snap, TikTok and Meta did not immediately respond to requests for comment.
The letter asks the company to provide:
- Policy definition of “deepfake” content, “non-consensual intimate images,” or similar terms.
- A description of the company’s policy and enforcement approach to non-consensual AI falsification of human bodies, non-nude images, altered clothing and “virtual undressing.”
- A description of our current content policy regarding edited media and explicit content, as well as internal guidance provided to moderators.
- How current policies regulate AI tools and image generators as they relate to suggestive or intimate content.
- What filters, guardrails or measures have been taken to prevent the generation and distribution of deepfakes.
- Mechanisms companies use to identify fake content and prevent it from being uploaded again.
- How to prevent users from profiting from the content.
- How platforms prevent themselves from monetizing non-AI-generated content.
- How a company’s terms of service can ban or suspend users who post deepfakes.
- What companies are doing to notify victims of non-consensual sexual deepfakes.
The letter was signed by Senators Lisa Blunt Rochester (D-Del.), Tammy Baldwin (D-Wis.), Richard Blumenthal (D-Conn.), Kirsten Gillibrand (DN.Y.), Mark Kelly (D-Ariz.), Ben Ray Luján (DN.M.), Brian Schatz (D-Hawaii), and Adam Schiff (D-Calif.).
The move comes just a day after xAI owner Elon Musk said he “I don’t know of any child nude pictures made by Grok. Later on Wednesday, California’s attorney general open an investigation to the xAI chatbot, following the pressure from governments around the world incensed by the lack of guardrails around Grok that allowed this to happen.
xAI already guarded who took action to remove “illegal content in X, including (CSAM) and non-consensual mudho,” although the company or Musk has addressed the fact that Grok allowed to generate such edits in the first place.
The problem is not limited to non-consensual manipulated sexual imagery. Although not all image generation and AI-based editing services allow users to “unmask” people, they can easily create fakes. To pick a few examples, OpenAI’s Sora 2 is reported allows users to create explicit videos featuring children; Google Nano Banana apparently if a picture showing Charlie Kirk being shot; and racist video created with Google’s AI video model gets millions of views on social media.
The problem is further complicated when Chinese image and video generators come into the picture. Many Chinese tech companies and apps – especially those linked to ByteDance – offer easy ways to edit faces, voices and videos, and their output has spread to Western social platforms. China has stronger synthetic content labeling requirements that the US does not have at the federal level, where the masses instead rely on fragmented and reinforced policies from the platform.
US lawmakers have passed several laws seeking to curb fake pornography, but their impact has been limited. The Take It Down Actwhich became a federal law in May, meant to criminalize the creation and dissemination of non-consensual, sexual imagery. But some provisions in the law making it difficult to hold platforms that produce images accountable, as they focus more on surveillance on individual users.
Meanwhile, some countries are trying to take matters into their own hands to protect consumers and voters. this week, New York Governor Kathy Hochul proposed the legislation which requires AI-generated content to be labeled as such, and prohibits non-consensual deepfakes in certain periods leading up to elections, including depictions of opposition candidates.