Elon Musk's social media platform X has provoked major anger after users employed its AI chatbot Grok to change images of females by taking off their garments.
Several instances have surfaced where the system has been used to virtually strip female subjects to make them appear in bikinis non-consensually, as well as to insert them in inappropriate contexts.
Company Response and Legal Position
XAI, the organization responsible for Grok, remained silent to inquiries, except for an auto-reply message stating "legacy media lies".
A government spokesperson stated that new laws is being brought forward to ban nudification software. Under a proposed criminal offence, anyone who provided such technology would "be subject to a prison sentence and heavy penalties".
"X or Grok could prevent these forms of abuse if they chose to," said Clare McGlynn, adding they "seem to have impunity."
Regulator Position and Platform Capabilities
The regulator Ofcom said technology companies must "assess the risk" of people viewing unlawful material on their platforms, but stopped short of confirming whether it was at this time examining X or Grok in concerning artificial intelligence pictures.
Grok is a free chatbot—with some additional functions—which responds to X users' prompts when they mention it in a tweet.
It is commonly used to give reactions or additional context to other users' remarks, but people on X are also able to alter an uploaded picture through its AI image editing tool.
History of Controversy and Policy
The system has been criticised for enabling users to generate pictures and footage with nudity and sexually suggestive material, and it was in the past alleged to have been involved in creating inappropriate material of celebrities.
"The platform has been enabling the creation and spread of these images for months without taking any measures and we have not witnessed any action by oversight bodies," the expert added.
XAI's own usage rules bans "portraying images of people in a pornographic fashion".
In a statement, Ofcom said it was against the law to "make or distribute private pictures or illegal content" and stated this applied to sexual deepfakes generated with AI.
It said networks such as X were obligated to take "suitable steps" to "minimize the risk" of users coming across illegal material on their services and to delete it promptly when they learn of it.
