The Internet Watch Foundation (IWF) says its analysts have discovered “criminal images” of girls aged 11 to 13 that “appear to have been created” with the help of Grok.
The artificial intelligence tool belongs to Elon Musk's company xAI. It can be accessed either through the website and app or through the X social media platform.
The IWF said it found “sexualized and topless images of girls” on a “dark web forum” where users claimed to have used Grock to create the images.
The BBC has approached X and xAI for comment.
The IWF's Ngaire Alexander told the BBC that tools like Grok now risk “bringing AI-generated sexual images of children into the mainstream.”
He said under UK law the material would be classified as Category C material – the lowest level of serious criminal material.
But he said the user who uploaded it then used another artificial intelligence tool not created by xAI to create the Category A image, the most serious category.
“We are extremely concerned about the ease and speed with which people can create photorealistic child sexual abuse material (CSAM),” he said.
Charity that seeks to remove child sexual abuse material from the Internet, operates a hotline to report suspected CSAM, and employs analysts who evaluate the legality and seriousness of this material.
Its analysts found the material on the darknet – no images were found on social network X.
Ofcom previously contacted X and xAI following reports that Grok could be used to create “sexualized images of children” and strip women.
The BBC has seen several examples on social media platform X of people asking the chatbot to alter real-life images to make women appear in bikinis without their consent, as well as putting them in sexual situations.
The IWF said it had received reports of such images on X, but until now they had not been assessed as meeting the legal definition of CSAM.
In a previous statement, X said: “We take action against illegal content on X, including CSAM, by removing it, permanently suspending accounts, and working with local authorities and law enforcement as necessary.
“Anyone who uses Grok or encourages it to create illegal content will suffer the same consequences as if they had uploaded illegal content.”
