The social media platform X has sparked widespread outrage after individuals used its artificial intelligence assistant Grok to manipulate photos of females by taking off their clothing.
Numerous cases have been seen where the tool has been used to virtually strip female subjects to render them in bikinis without consent, as well as to put them in inappropriate contexts.
Developer Response and Regulatory Stance
XAI, the firm responsible for Grok, did not respond to requests for comment, other than an auto-reply response stating "traditional news outlets lies".
A Home Office spokesperson stated that new laws is being brought forward to prohibit nudification tools. Under a proposed criminal offence, anyone who distributed such technology would "receive a prison sentence and substantial penalties".
"The platform is capable of halting these types of misuse if they wanted to," said Clare McGlynn, adding they "operate with a lack of consequences."
Regulator Stance and Platform Functionality
The regulator Ofcom said tech firms must "evaluate the danger" of people seeing illegal content on their services, but stopped short of confirming whether it was presently looking into X or Grok in concerning AI-generated images.
Grok is a free artificial intelligence—with some additional functions—which reacts to X users' queries when they mention it in a tweet.
It is often used to give reactions or further context to other posters' remarks, but people on X are also able to alter an posted image through its artificial intelligence picture modification tool.
Background of Controversy and Policy
The tool has been condemned for permitting users to create pictures and footage with explicit content and sexually suggestive material, and it was in the past alleged to have been involved in making sexually explicit content of public figures.
"The platform has been allowing the production and sharing of these pictures for months without taking any action and we have not seen any challenge by oversight bodies," the expert added.
XAI's own acceptable use policy bans "showing images of persons in a sexually explicit way".
In a comment, Ofcom said it was illegal to "create or circulate deepfakes or child sexual abuse material" and confirmed this covered sexual deepfakes generated with AI.
It said platforms such as X were obligated to take "necessary steps" to "lessen the risk" of users encountering unlawful content on their services and to delete it quickly when they are notified of it.
