Character.AI bans under-18s from interacting with chatbots after teen suicide

Character.AI isbanning minors from using its chatbots amid growing concerns about the effects of artificial intelligence (AI) conversations on children.

The company is facing several lawsuits over child safety, including by the mother of a teenager who says the company’s chatbots pushed her teenage son to kill himself.

Character Technologies stated that users under 18 won't be able to engage in open-ended conversations with its chatbot characters, and the bot will implement a 2-hour usage limit by November 25.

The company lets users create or interact with customisable characters that "feel alive and humanlike" for a range of activities like playing or doing mock job interviews.

Character.AI said it will be rolling out age-verification functions to help determine which users are under 18. A growing number of tech platforms are turning to age checks to keep children from accessing tools that aren't safe for them.

But these are imperfect, and many kids find ways to get around them. Face scans, for instance, can't always tell if someone is 17 or 18. And there are privacy concerns around asking people to upload government IDs.

Character.AI added that it is working on new features for kids, such as creating videos, stories, and streams with AI characters, as well as an AI safety lab. The company is also setting up an AI safety lab

Meetali Jain, executive director of the Tech Justice Law Project, saidthe move by Character.AI “still [has] a lot of details left open”.

“They have not addressed how they will operationalise age verification, how they will ensure their methods are privacy preserving, nor have they addressed the possible psychological impact of suddenly disabling access to young users, given the emotional dependencies that have been created,” Jain said.

“Moreover, these changes do not address the underlying design features that facilitate these emotional dependencies – not just for children, but also for people over the age of 18 years.”

More than 70 per cent of teens have used AI companions and half use them regularly, according to a recent studyfrom Common Sense Media, a group that studies and advocates for using screens and digital media sensibly.