The social media platform X has provoked significant anger after people utilized its AI chatbot Grok to alter photos of females by stripping their attire.
Multiple examples have surfaced where the system has been employed to digitally undress women to make them appear in swimwear without permission, as well as to put them in inappropriate scenarios.
Company Reaction and Legal Stance
XAI, the company that created Grok, remained silent to inquiries, aside from an standard message stating "legacy media lies".
A government representative stated that new laws is being brought forward to outlaw nudification tools. Under a new criminal offence, anyone who distributed such tools would "receive a prison sentence and heavy fines".
"X or Grok is capable of halting these forms of abuse if they wanted to," said a law professor, adding they "operate with a lack of consequences."
Oversight Body Stance and Tool Capabilities
The regulator Ofcom said technology companies must "consider the risk" of people accessing illegal content on their platforms, but stopped short of confirming whether it was presently examining X or Grok in relation to AI-generated images.
Grok is a no-cost chatbot—with some paid-for features—which reacts to X users' queries when they tag it in a post.
It is often used to give reactions or further background to other posters' comments, but people on X are also can alter an posted image through its artificial intelligence picture modification tool.
History of Controversy and Policy
The system has been faulted for allowing users to create pictures and footage with explicit content and sexually suggestive content, and it was earlier accused of making sexually explicit content of public figures.
"The platform has been enabling the production and sharing of these images for months without taking any measures and we have not seen any action by regulators," Professor McGlynn added.
XAI's own acceptable use policy prohibits "showing images of individuals in a sexually explicit way".
In a comment, Ofcom said it was against the law to "create or distribute private pictures or child sexual abuse material" and confirmed this covered sexual deepfakes created with AI.
It said platforms such as X were mandated to take "suitable actions" to "reduce the danger" of users coming across unlawful material on their platforms and to delete it promptly when they become aware of it.
