🔗 Share this article

The popular AI conversation site Character.ai is introducing a major restriction that will block young users from having discussions with its artificial intelligence-driven companions.

This action comes after growing backlash regarding the types of interactions young people were having with the AI entities.

Lawsuits

The service, which started in 2021 and attracts numerous people, is confronting various court cases in the US.

Parents have initiated cases, among them involving the unfortunate passing of a teenager, with some calling the platform a "serious risk" to teenagers.

New Restrictions

Beginning November 25th, users under 18 will only have permission to produce videos such as videos with their characters, instead of having discussions as they now do.

Digital protection campaigners have welcomed the move but expressed that the capability should never have been offered to children in the beginning.

Company Statement

"Today's announcement is a extension of our fundamental approach that we need to keep developing the safest AI platform on the globe for recreational use," said Character.ai boss Karandeep Anand.

He continued that AI safety is "a moving target" but something the company had taken an "forceful" approach to, with parental controls and protective guardrails.

Specialist Views

Digital protection body Internet Matters welcomed the announcement, but emphasized that safety measures ought to have been incorporated from the start.

"Studies we've conducted indicates that children are exposed to risky information and put at risk when using AI, including AI chatbots," the group said.

Earlier Concerns

Character.ai has been questioned in the past for including potentially harmful or concerning chatbots that young users could talk to.

Virtual profiles mimicking UK youths were found on the site before being deleted.

Later, researchers identified a chatbot modeled after convicted sex offender which had logged thousands of chats with users.

Sector Consequences

Social media expert Matt Navarra called the announcement as a "wake-up call" for the AI industry, which is moving "from unrestricted development to responsive measures".

"When a platform that creates a youth-focused platform still then pulls the plug, it's demonstrating that moderated conversations are insufficient when the tech's emotional pull is intense," he remarked.

Future Directions

The platform says its new focus will be on providing "deeper gameplay" and "character-driven stories" for young users.

Enhanced age checking systems will also be implemented, and the platform will support a novel AI protection study center.

Dr Nomisha Kurian, who has studied AI safety, said it was "a prudent action" to control teens from using chatbots.

"It helps to distinguish creative play from more private, psychologically vulnerable interactions," she elaborated.

"This represents so important for young users still learning to navigate emotional and digital restrictions."

She mentioned that Character.ai's revised approaches might reflect a "maturing phase" in the AI industry, where youth protection is gradually being acknowledged as an "immediate concern" for responsible innovation.