Elon Musk’s xAI has restricted the image-generating capabilities of its artificial intelligence chatbot Grok to paying subscribers after it was widely condemned for being used to create non-consensual sexualized images of real women and children.
“Image generation and editing are currently limited to paid subscribers,” Grok via X on Friday. This restriction means that the vast majority of users can no longer access the feature. Paid, verified subscribers with credit card details can still do so, but in theory they could be more easily identified if the feature is abused.
Some experts are not convinced the new restrictions will solve the widespread problem.
Henry Ajder, a British deepfake expert, said: “Given how easy it is to provide false information and use improvised payment methods, it is also unconvincing that providing user details and payment methods will help identify the perpetrators.” wealth. “The logic here is also reactive: it should help identify offenders after the content is generated, but it does not represent any consistency or meaningful constraints on the model itself.”
X did not immediately respond to Fortune’s request for comment, and xAI responded with an automated message: “Legacy media lies.”
Over the past week, real women have been targeted at scale, with users altering photos to remove their clothes, put subjects in bikinis, or place them in explicit sexual scenes without their consent. Some victims said the trend left them feeling violated and upset, with many saying their reports to X went unanswered and the images remained on the platform.
The researchers say the scale at which Grok produces and shares images is unprecedented because, unlike other AI bots, Grok essentially has a distribution system built into the X platform.
One researcher’s analysis is publisher Bloomberg, It is estimated that X has become the website with the most deepfakes in the last week. Genevieve Oh, a social media and deepfake researcher, conducted a 24-hour analysis of images posted to X by the @Grok account, Discover chat blogApproximately 6,700 sexually suggestive or nude images are generated every hour. By comparison, the other five leading pornographic deepfake sites posted an average of 79 new AI-stripped images per hour during the same period. Oh’s research also found that sexual content dominated Grok’s output, accounting for 85% of all images generated by the chatbot.
Ashley St. Clair, a conservative commentator and mother of one of Musk’s children, was among those affected by the images. st clair told wealth The user was turning images on her X profile into explicit AI-generated photos, some of which she said depicted her as a minor. After speaking out against the images and raising concerns about deepfakes from minors, St Clair also said X removed her verified paid subscriber status without notifying her or refunding her $8 monthly fee.
“Limiting it to paid subscribers only shows that they are going to double down on their efforts and place an undue burden on victims to report them to law enforcement and use their resources to track these people down,” Ashley St. Clair said of the recent restrictions. “It’s also a money grab.”
St. Clair said many of the accounts targeted against her were already verified users: “It had no effect at all,” she said. “This is just in anticipation of more law enforcement investigations regarding the generation of Grok images.”
regulatory pressure
The move to limit Grok’s capabilities comes amid growing pressure from regulators around the world. In the UK, Prime Minister Keir Starmer Show that he is open banned the platform entirely and described its content as “shameful” and “disgusting”. regulatory agencies in India, Malaysia and France An investigation or inquiry was also launched.
European Commission Ordered X on Thursday In an effort to preserve all internal documents and data related to Grok, an investigation into the platform’s content moderation practices has been stepped up after the spread of non-consensual sexually explicit deepfakes was described as “illegal”, “shocking” and “disgusting”.
Experts say the new restrictions may not satisfy regulators’ concerns: “This approach is a blunt approach that does not address the root of the Grok Alliance’s problems and may not achieve agreement with regulators,” Ajder said. “Restricting functionality to paid subscribers will not prevent the generation of this type of content; a one-month subscription is not a reliable solution.”
In the United States, the situation could also test existing laws such as Section 230 of the Communications Decency Act, which exempts online providers from liability for user-created content.
Riana Pfefferkorn of the Institute for Human-Centered Artificial Intelligence at Stanford University previously stated wealth Responsibilities surrounding AI-generated images are murky. “For the first time, we have a situation where the platforms themselves are generating non-consensual pornographic content for adults and minors at scale,” she said. “The CSAM law poses the greatest potential liability risk from a liability perspective and a PR perspective.”
Musk has previously said that “anyone who uses Grok to create illegal content will suffer the same consequences as uploading illegal content.” However, it is unclear how accounts will be held accountable.