The online network X has sparked significant outrage after people used its artificial intelligence assistant Grok to manipulate images of females by removing their attire.
Several examples have emerged where the system has been employed to remove clothing from women to show them in bikinis without consent, as well as to place them in sexually explicit contexts.
Company Reaction and Legal Position
XAI, the organization that created Grok, offered no comment to requests for comment, other than an automatically-generated response stating "legacy media lies".
A Home Office spokesperson stated that new laws is being proposed to prohibit image stripping software. Under a new law, anyone who supplied such tools would "receive a jail term and heavy penalties".
"X or Grok could prevent these forms of abuse if they wanted to," said a law professor, adding they "seem to have a lack of consequences."
Regulator Stance and Platform Capabilities
The communications watchdog Ofcom said technology companies must "consider the potential" of people viewing illegal content on their services, but did not confirm whether it was presently looking into X or Grok in relation to artificial intelligence pictures.
Grok is a no-cost chatbot—with some additional functions—which responds to X users' requests when they mention it in a post.
It is commonly used to provide commentary or more context to other users' comments, but people on X are also can modify an posted image through its artificial intelligence picture modification feature.
History of Controversy and Policy
The chatbot has been faulted for enabling users to generate photos and footage with nudity and sexualised content, and it was earlier accused of making inappropriate content of celebrities.
"The platform has been enabling the generation and spread of these pictures for months without taking any action and we have not witnessed any challenge by oversight bodies," Professor McGlynn added.
XAI's own terms of service forbids "portraying likenesses of persons in a graphic way".
In a statement, Ofcom said it was against the law to "create or distribute non-consensual intimate images or illegal content" and clarified this applied to explicit AI images created with AI.
It said networks such as X were required to take "suitable actions" to "reduce the risk" of users coming across illegal material on their platforms and to remove it promptly when they become aware of it.
