The online network X has caused widespread outrage after individuals utilized its AI chatbot Grok to change photos of women by taking off their clothing.
Multiple examples have surfaced where the technology has been used to virtually strip female subjects to make them appear in bikinis without permission, as well as to put them in inappropriate contexts.
Company Response and Regulatory Position
XAI, the firm responsible for Grok, offered no comment to requests for comment, except for an automatically-generated response stating "traditional news outlets lies".
A government spokesperson stated that legislation is being proposed to ban nudification tools. Under a proposed law, anyone who provided such technology would "be subject to a jail term and substantial penalties".
"The platform has the ability to stop these forms of abuse if they wanted to," said a law professor, adding they "seem to have a lack of consequences."
Regulator Position and Tool Functionality
The regulator Ofcom said tech firms must "assess the risk" of people seeing unlawful material on their platforms, but did not specify whether it was at this time investigating X or Grok in relation to artificial intelligence images.
Grok is a no-cost AI assistant—with some additional functions—which responds to X users' queries when they tag it in a tweet.
It is commonly used to provide reactions or further background to other users' comments, but people on X are also capable of modify an uploaded image through its AI image editing feature.
Background of Criticism and Policy
The system has been criticised for permitting users to produce pictures and clips with nudity and sexually suggestive content, and it was earlier accused of making inappropriate content of celebrities.
"X has been allowing the generation and sharing of these pictures for months without taking any action and we have not witnessed any intervention by regulators," Professor McGlynn added.
XAI's own usage rules forbids "showing likenesses of individuals in a pornographic manner".
In a statement, Ofcom said it was against the law to "generate or distribute non-consensual intimate images or child sexual abuse material" and clarified this included explicit AI images generated with AI.
It said networks such as X were required to take "necessary measures" to "reduce the danger" of users coming across illegal content on their services and to delete it swiftly when they become aware of it.
