The social media platform X has caused major outrage after users utilized its AI chatbot Grok to alter photos of women by stripping their garments.
Several instances have been seen where the technology has been used to digitally undress female subjects to make them appear in bikinis without consent, as well as to put them in sexually explicit contexts.
Company Reaction and Legal Stance
XAI, the company behind Grok, did not respond to requests for comment, aside from an automatically-generated response stating "traditional news outlets lies".
A government representative stated that new laws is being introduced to prohibit nudification software. Under a upcoming criminal offence, anyone who distributed such technology would "face a prison sentence and substantial fines".
"The platform could prevent these types of abuse if they wanted to," said a law professor, adding they "seem to have impunity."
Oversight Body Position and Platform Functionality
The communications watchdog Ofcom said platforms must "assess the risk" of people viewing unlawful material on their networks, but stopped short of confirming whether it was at this time looking into X or Grok in regarding AI-generated images.
Grok is a free chatbot—with some premium features—which responds to X users' prompts when they tag it in a post.
It is frequently used to offer reactions or additional background to other users' remarks, but people on X are also capable of alter an uploaded image through its artificial intelligence picture modification tool.
Background of Criticism and Policy
The system has been faulted for allowing users to produce pictures and clips with explicit content and sexualised content, and it was in the past accused of making sexually explicit material of celebrities.
"X has been permitting the generation and sharing of these pictures for a long time without taking any steps and we have not witnessed any action by oversight bodies," Professor McGlynn added.
XAI's own acceptable use policy bans "showing images of persons in a pornographic way".
In a statement, Ofcom said it was unlawful to "generate or circulate deepfakes or illegal content" and confirmed this included sexual deepfakes generated with AI.
It said networks such as X were mandated to take "suitable actions" to "reduce the danger" of users coming across unlawful content on their platforms and to remove it quickly when they become aware of it.
