The social media network X has provoked significant backlash after people employed its artificial intelligence assistant Grok to change pictures of females by taking off their clothing.
Several examples have emerged where the tool has been employed to remove clothing from female subjects to show them in bikinis without consent, as well as to put them in sexually explicit situations.
Company Reaction and Legal Position
XAI, the company behind Grok, did not respond to inquiries, aside from an standard response stating "legacy media lies".
A Home Office official stated that legislation is being introduced to ban image stripping software. Under a new criminal offence, anyone who supplied such technology would "face a jail term and substantial fines".
"The platform could prevent these forms of misuse if they wanted to," said a law professor, adding they "appear to enjoy impunity."
Regulator Position and Tool Functionality
The regulator Ofcom said tech firms must "assess the danger" of people viewing unlawful material on their services, but did not specify whether it was at this time examining X or Grok in regarding AI-generated images.
Grok is a free chatbot—with some paid-for functions—which reacts to X users' requests when they reference it in a message.
It is often used to give reactions or further context to other users' comments, but people on X are also able to alter an posted photo through its artificial intelligence picture modification feature.
Background of Controversy and Rules
The system has been condemned for enabling users to create images and videos with nudity and sexually suggestive material, and it was previously alleged to have been involved in creating graphic material of celebrities.
"The platform has been allowing the production and distribution of these pictures for a long time without taking any steps and we have not seen any intervention by regulators," Professor McGlynn added.
XAI's own usage rules prohibits "showing likenesses of people in a graphic fashion".
In a comment, Ofcom said it was against the law to "create or share deepfakes or illegal content" and stated this covered explicit AI images created with AI.
It said platforms such as X were obligated to take "appropriate measures" to "reduce the risk" of users encountering illegal material on their platforms and to delete it swiftly when they learn of it.
