The social media network X has caused significant anger after individuals utilized its artificial intelligence assistant Grok to change photos of females by removing their garments.
Several cases have emerged where the system has been used to remove clothing from female subjects to make them appear in bikinis non-consensually, as well as to place them in inappropriate scenarios.
Developer Reaction and Regulatory Position
XAI, the company that created Grok, offered no comment to requests for comment, aside from an standard response stating "traditional news outlets lies".
A government official stated that new laws is being brought forward to outlaw nudification technology. Under a proposed law, anyone who distributed such tools would "be subject to a jail term and significant penalties".
"X or Grok has the ability to stop these forms of misuse if they wanted to," said a law professor, adding they "operate with a lack of consequences."
Regulator Stance and Platform Capabilities
The communications watchdog Ofcom said platforms must "evaluate the danger" of people seeing unlawful material on their services, but did not specify whether it was presently investigating X or Grok in regarding artificial intelligence images.
Grok is a free chatbot—with some premium features—which responds to X users' prompts when they reference it in a tweet.
It is frequently used to give reactions or more context to other posters' remarks, but people on X are also capable of alter an posted photo through its AI image editing tool.
Background of Criticism and Rules
The system has been condemned for permitting users to generate images and clips with explicit content and sexually suggestive material, and it was in the past alleged to have been involved in generating sexually explicit material of public figures.
"The platform has been enabling the generation and sharing of these images for months without taking any steps and we have not witnessed any action by regulators," the expert added.
XAI's own terms of service prohibits "depicting images of individuals in a sexually explicit way".
In a comment, Ofcom said it was against the law to "make or distribute private pictures or child sexual abuse material" and stated this applied to sexual deepfakes made with AI.
It said services such as X were required to take "appropriate actions" to "minimize the risk" of users coming across unlawful content on their services and to delete it quickly when they learn of it.
