Elon Musk’s X must urgently deal with its in-built artificial intelligence Grok being used to generate “absolutely appalling” fake sexualised images, Technology Secretary Liz Kendall has said.
Ofcom raised “serious concerns” on Monday with the social media platform’s AI tool, as it can be used to create “undressed images of people and sexualised images of children”.
Since the start of the new year, X users – mainly women – have reported that accounts have used Grok to generate images of them without clothing.
There are also several cases where Grok has created sexualised images of children, according to analysis by news agency Reuters.
Technology Secretary Liz Kendall. File pic: PA
“What we have been seeing online in recent days has been absolutely appalling, and unacceptable in decent society,” said Ms Kendall.
“No one should have to go through the ordeal of seeing intimate deepfakes of themselves online.
“We cannot and will not allow the proliferation of these demeaning and degrading images, which are disproportionately aimed at women and girls.
“X needs to deal with this urgently. It is absolutely right that Ofcom is looking into this as a matter of urgency and it has my full backing to take any enforcement action it deems necessary.”
Sky News has seen multiple users sharing disgust, after non-consensual deepfakes of themselves in bikinis were shared on the app.
A statement shared on the X’s official safety account said: “We take action against illegal content on X, including Child Sexual Abuse Material (CSAM), by removing it, permanently suspending accounts, and working with local governments and law enforcement as necessary.”
A post on the Grok X account previously said there had been “isolated cases where users prompted for and received AI images depicting minors in minimal clothing.
“xAI has safeguards, but improvements are ongoing to block such requests entirely,” it added.
Under the Online Safety Act, which began being fully enforced in July, it is an offence to share intimate imagery of someone without their permission, even if it is made using AI.
On Friday, French ministers reported sexually explicit content generated by Grok on X to prosecutors, saying that the “sexual and sexist” content was “manifestly illegal”.
Can X be forced to take action?
Under the UK’s online safety act, platforms that allow illegal content like sexual deepfakes to be shared online can be fined up to £18 million or 10 percent of their qualifying worldwide revenue, whichever is greater.
That applies even if the company is based outside the UK, like X, but has a lot of UK users or the potential to harm UK users.
The EU has similar regulation, called the Digital Services Act, and can also fine companies.
In fact, just last month X was handed a €120m (£104m) fine for breaching the DSA.
In the USA, some groups are calling on authorities to take action against xAI.
The Consumer Federation of America (CFA) on Monday wrote to the Federal Trade Commission, the Federal Communications Commission, the Department of Justice, and all Attorneys General, demanding swift action.
“xAI is purposefully and recklessly endangering people on their platform and hoping to avoid accountability just because it’s ‘AI’,” said Ben Winters, Director of AI and Data Privacy at CFA.
“AI is no different than any other product – the company has chosen to break the law and must be held accountable.”
