The social media network X has sparked widespread anger after individuals used its artificial intelligence assistant Grok to change pictures of females by stripping their garments.
Multiple examples have emerged where the system has been used to remove clothing from female subjects to show them in swimwear without consent, as well as to place them in sexually explicit scenarios.
Company Response and Legal Position
XAI, the organization that created Grok, offered no comment to inquiries, except for an automatically-generated message stating "traditional news outlets lies".
A government representative stated that new laws is being introduced to prohibit image stripping software. Under a new criminal offence, anyone who supplied such technology would "be subject to a prison sentence and significant fines".
"X or Grok has the ability to stop these types of misuse if they wanted to," said Clare McGlynn, adding they "seem to have a lack of consequences."
Oversight Body Position and Platform Functionality
The regulator Ofcom said technology companies must "assess the danger" of people viewing illegal content on their services, but did not specify whether it was currently looking into X or Grok in regarding artificial intelligence pictures.
Grok is a no-cost artificial intelligence—with some premium features—which answers X users' requests when they reference it in a message.
It is often used to provide commentary or additional context to other users' comments, but people on X are also can alter an uploaded picture through its artificial intelligence picture modification tool.
Background of Controversy and Rules
The tool has been faulted for permitting users to generate images and videos with explicit content and sexualised content, and it was earlier accused of generating inappropriate material of celebrities.
"X has been enabling the production and spread of these images for months without taking any action and we have not witnessed any intervention by regulators," Professor McGlynn added.
XAI's own usage rules bans "portraying images of persons in a sexually explicit manner".
In a statement, Ofcom said it was illegal to "make or circulate private pictures or illegal content" and clarified this covered sexual deepfakes created with AI.
It said platforms such as X were obligated to take "necessary actions" to "reduce the risk" of users viewing illegal material on their platforms and to remove it quickly when they become aware of it.
