The online network X has provoked widespread backlash after individuals used its artificial intelligence assistant Grok to manipulate images of females by stripping their attire.
Multiple instances have been seen where the tool has been used to remove clothing from women to make them appear in bikinis without permission, as well as to place them in inappropriate scenarios.
Company Response and Legal Stance
XAI, the organization that created Grok, did not respond to requests for comment, other than an auto-reply response stating "legacy media lies".
A government official stated that legislation is being brought forward to ban image stripping technology. Under a proposed law, anyone who supplied such tools would "be subject to a prison sentence and significant penalties".
"X or Grok could prevent these types of abuse if they chose to," said a law professor, adding they "appear to enjoy a lack of consequences."
Oversight Body Position and Platform Capabilities
The regulator Ofcom said tech firms must "evaluate the potential" of people seeing illegal content on their platforms, but did not specify whether it was currently investigating X or Grok in concerning artificial intelligence pictures.
Grok is a free AI assistant—with some additional features—which responds to X users' requests when they tag it in a tweet.
It is frequently used to provide commentary or additional background to other posters' comments, but people on X are also can edit an uploaded photo through its artificial intelligence picture modification tool.
Background of Controversy and Policy
The tool has been condemned for permitting users to produce photos and footage with nudity and sexualised content, and it was earlier alleged to have been involved in generating graphic content of celebrities.
"The platform has been permitting the production and spread of these pictures for a long time without taking any action and we have not seen any action by oversight bodies," the expert added.
XAI's own usage rules bans "portraying images of people in a graphic way".
In a comment, Ofcom said it was unlawful to "create or circulate deepfakes or illegal content" and confirmed this applied to sexual deepfakes generated with AI.
It said services such as X were mandated to take "suitable measures" to "reduce the danger" of users encountering unlawful material on their platforms and to delete it promptly when they learn of it.
