The social media platform X has caused major anger after individuals employed its AI chatbot Grok to alter photos of women by taking off their garments.
Numerous instances have been seen where the technology has been used to virtually strip female subjects to render them in bikinis non-consensually, as well as to put them in sexualized contexts.
Developer Response and Regulatory Stance
XAI, the firm behind Grok, did not respond to requests for comment, aside from an automatically-generated response stating "traditional news outlets lies".
A Home Office representative stated that legislation is being brought forward to outlaw nudification software. Under a new criminal offence, anyone who distributed such technology would "face a prison sentence and substantial penalties".
"The platform could prevent these types of abuse if they desired to," said a law professor, adding they "seem to have impunity."
Regulator Position and Platform Capabilities
The communications watchdog Ofcom said technology companies must "consider the risk" of people viewing illegal content on their services, but stopped short of confirming whether it was presently looking into X or Grok in concerning AI-generated images.
Grok is a free artificial intelligence—with some additional functions—which reacts to X users' prompts when they mention it in a post.
It is frequently used to give reactions or further context to other users' remarks, but people on X are also capable of alter an uploaded image through its AI image editing tool.
History of Criticism and Rules
The chatbot has been criticised for allowing users to create photos and videos with explicit content and sexually suggestive content, and it was previously accused of creating graphic content of celebrities.
"X has been enabling the production and sharing of these pictures for a long time without taking any steps and we have not witnessed any challenge by regulators," the expert added.
XAI's own terms of service bans "depicting likenesses of persons in a sexually explicit way".
In a statement, Ofcom said it was against the law to "create or circulate private pictures or illegal content" and confirmed this included explicit AI images created with AI.
It said platforms such as X were required to take "suitable actions" to "reduce the risk" of users encountering illegal material on their platforms and to take down it swiftly when they are notified of it.
