Litigation and security issues
Character.AI was founded in 2021 by Noam Shazir and Daniel De Freitas, two former Google engineers, and has raised nearly $200 million from investors. Last year, Google agreed to pay about $3 billion to license Character.AI's technology, and Shazir and De Freitas back on Google.
But the company is now facing multiple lawsuits alleging its technology contributed to the deaths of teenagers. Last year, the family of 14-year-old Sewell Setzer III sued The character.AI blames the company for his death. Setzer committed suicide after frequently texting and talking to one of the platform's chatbots. The company is facing additional claimsincluding a representative of a Colorado family whose 13-year-old daughter Juliana Peralta committed suicide in 2023 after using the platform.
In December character.AI announced changes, including improved detection of infringing content and revised terms of service, but these measures did not restrict minor users' access to the platform. Other AI chatbot services such as OpenAI's ChatGPT also come under scrutiny for the impact of their chatbots on young users. In September, OpenAI introduced parental controls designed to give parents more information about how their children use the service.
These cases caught the attention of government officials, which likely prompted Feature.AI to announce changes to chat access for children under 18. Steve Padilla, a California Senate Democrat who introduced the security bill, told The New York Times that “there are a growing number of stories about what can go wrong. It's important to put in place reasonable protections so we can protect the most vulnerable people.”
On Tuesday, Senators Josh Hawley and Richard Blumenthal presented a bill to prohibit the use of AI companions by minors. Also this month, California Gov. Gavin Newsom signed the lawwhich takes effect on January 1 and requires artificial intelligence companies to install security guardrails for chatbots.
