The social media platform X has provoked significant outrage after people employed its artificial intelligence assistant Grok to change images of women by removing their clothing.
Several cases have surfaced where the tool has been employed to digitally undress women to make them appear in swimwear non-consensually, as well as to insert them in sexually explicit situations.
Company Reaction and Regulatory Stance
XAI, the organization that created Grok, did not respond to inquiries, other than an auto-reply response stating "traditional news outlets lies".
A government official stated that new laws is being introduced to ban nudification software. Under a new law, anyone who provided such tools would "face a prison sentence and significant fines".
"The platform has the ability to stop these types of abuse if they desired to," said Clare McGlynn, adding they "operate with impunity."
Oversight Body Position and Platform Functionality
The regulator Ofcom said platforms must "assess the risk" of people accessing illegal content on their networks, but did not specify whether it was at this time looking into X or Grok in regarding artificial intelligence pictures.
Grok is a free chatbot—with some premium functions—which reacts to X users' requests when they tag it in a post.
It is commonly used to offer reactions or more background to other users' comments, but people on X are also able to alter an uploaded picture through its AI image editing feature.
History of Criticism and Rules
The chatbot has been faulted for enabling users to create images and footage with nudity and sexually suggestive content, and it was earlier alleged to have been involved in generating sexually explicit content of public figures.
"The platform has been allowing the generation and distribution of these images for a long time without taking any measures and we have yet to see any intervention by oversight bodies," Professor McGlynn added.
XAI's own usage rules bans "portraying images of individuals in a sexually explicit fashion".
In a comment, Ofcom said it was illegal to "make or distribute non-consensual intimate images or child sexual abuse material" and confirmed this applied to explicit AI images created with AI.
It said services such as X were mandated to take "necessary actions" to "reduce the danger" of users encountering illegal content on their services and to remove it promptly when they become aware of it.
