🔗 Share this article

The popular virtual character service Character.ai is introducing a major policy change that will block teenagers from engaging in conversations with its AI-powered companions.

This decision is in response to intense criticism regarding the types of interactions young people were engaging in with the virtual friends.

Lawsuits

The company, which was founded in 2021 and has numerous people, is confronting multiple lawsuits in the United States.

Parents have initiated cases, among them concerning the tragic death of a minor, with some calling the platform a "immediate threat" to teenagers.

Changed Rules

Starting from November 25th, users under 18 will only be able to create material such as videos with their characters, instead of engaging in conversations as they presently are able to.

Online safety organizations have applauded the decision but stated that the function should never have been offered to children in the first place.

Official Response

"This decision is a part of our general belief that we need to keep building the safest AI platform on the globe for entertainment purposes," said Character.ai leader Karandeep Anand.

He added that AI safety represents "a moving target" but something the company had taken an "proactive" approach to, with guardian features and safety barriers.

Professional Perspectives

Digital protection body Internet Matters approved the announcement, but emphasized that security protocols ought to have been built in from the outset.

"Studies we've conducted shows that young people are confronted with dangerous material and placed in jeopardy when engaging with AI, particularly AI chatbots," the body stated.

Earlier Concerns

Character.ai has been criticized in the past for hosting potentially dangerous or offensive chatbots that children could engage with.

Avatars simulating British teenagers were discovered on the platform before being deleted.

Subsequently, researchers discovered a chatbot based on convicted sex offender which had accumulated multiple interactions with users.

Market Effects

Digital platform expert Matt Navarra characterized the development as a "wake-up call" for the AI industry, which is shifting "from unregulated creation to responsive measures".

"When a platform that creates a teen experience still then pulls the plug, it's demonstrating that moderated conversations are insufficient when the system's emotional engagement is strong," he commented.

Coming Changes

The platform states its updated direction will be on providing "deeper gameplay" and "role-play storytelling" for teenagers.

Updated identity confirmation processes will also be introduced, and the organization will support a fresh artificial intelligence security investigation facility.

An AI safety researcher, who has researched AI safety, said it was "a sensible move" to limit minors from using chatbots.

"It aids to differentiate artistic expression from more private, emotionally sensitive interactions," she explained.

"This constitutes so crucial for teenage users who are learning to manage emotional and digital limits."

She mentioned that Character.ai's revised approaches might reflect a "developing stage" in the AI industry, where youth protection is progressively being understood as an "urgent priority" for ethical development.