The social media platform X has sparked widespread backlash after users utilized its artificial intelligence assistant Grok to change images of women by stripping their clothing.
Several cases have emerged where the system has been used to digitally undress female subjects to make them appear in bikinis without permission, as well as to insert them in inappropriate contexts.
Developer Reaction and Legal Position
XAI, the company that created Grok, offered no comment to requests for comment, aside from an automatically-generated message stating "legacy media lies".
A Home Office official stated that legislation is being introduced to outlaw nudification software. Under a upcoming law, anyone who supplied such tools would "face a jail term and substantial penalties".
"X or Grok has the ability to stop these types of misuse if they chose to," said Clare McGlynn, adding they "seem to have a lack of consequences."
Oversight Body Position and Tool Functionality
The regulator Ofcom said platforms must "evaluate the danger" of people accessing unlawful material on their services, but stopped short of confirming whether it was at this time looking into X or Grok in regarding AI-generated images.
Grok is a free artificial intelligence—with some additional features—which answers X users' prompts when they reference it in a tweet.
It is often used to give commentary or more background to other posters' remarks, but people on X are also capable of edit an posted picture through its artificial intelligence picture modification tool.
Background of Criticism and Policy
The tool has been faulted for allowing users to produce photos and clips with explicit content and sexually suggestive material, and it was previously accused of creating graphic material of public figures.
"X has been permitting the creation and spread of these images for a long time without taking any action and we have not witnessed any action by oversight bodies," Professor McGlynn added.
XAI's own acceptable use policy bans "depicting images of people in a pornographic manner".
In a comment, Ofcom said it was unlawful to "generate or distribute non-consensual intimate images or illegal content" and clarified this included explicit AI images made with AI.
It said networks such as X were required to take "appropriate actions" to "reduce the risk" of users coming across illegal content on their platforms and to remove it swiftly when they learn of it.
