OpenAI says it does not allow its LLMs to be used in this way
When asked to comment on the sexual conversations detailed in the report, an OpenAI spokesperson said:
Minors deserve strong protections, and we have strict rules that developers must follow. We take enforcement action against developers when we determine that they have violated our policies, which prohibit any use of our services to exploit, threaten, or sexualize anyone under 18 years of age. These rules apply to every developer who uses our API, and we use classifiers to ensure that our services are not used to harm minors.
Interestingly, an OpenAI spokesperson told us that OpenAI is not directly involved with Alilo and that it has not seen any API activity on the Alilo domain. OpenAI is investigating the toy company to see if it routes traffic through OpenAI's API, the spokesperson said.
Alilo did not respond to Ars' request for comment before publication.
Companies that make products that use OpenAI technology and are marketed to children must comply with the Children's Online Privacy Protection Act (COPPA), when applicable, as well as any other relevant child protection, safety and privacy laws, and obtain parental consent, an OpenAI spokesperson said.
We've already seen how OpenAI deals with toy makers who violate its rules.
Last month, PIRG released the Toyland Challenges 2025 report (PDF), which details the sex-related conversations his testers were able to have with Kumm's teddy bear. A day later, OpenAI suspended FoloToy for violating its policies (the terms of the suspension were not disclosed), and FoloToy temporarily stopped selling.
The toy is back up for sale, and PIRG reported today that Kumma is no longer teaching kids how to light matches and get kinky.
But even toy companies that try to follow chatbot rules could be putting children at risk.
