Concerns mount over AI chatbot safety as parents sue platform over child's harm

ANKARA

The use of artificial intelligence (AI) chatbots by children has sparked growing concern in the US, as a Texas family filed a federal lawsuit against Character.AI, alleging the company’s chatbot contributed to their autistic son’s self-harm and violent behavior, media reports said on Sunday.

Character.AI, one of the most prominent platforms in the field, recently announced a new policy banning users under 18 from interacting with its chatbots, according to ABC News.

The company described the move as a “bold step forward” in protecting teenagers and vulnerable individuals.

However, for parents Mandi and Josh Furniss, the policy came too late.

According to the lawsuit and statements to the news outlet, their previously cheerful son began withdrawing from his family in 2023 after receiving a phone.

Over time, his behavior became increasingly dark and erratic, culminating in threats of violence and self-harm.

“When I saw the (chatbot) conversations, my first reaction was there’s a pedophile that’s come after my son,” Mandi Furniss told ABC News.

Screenshots included in the lawsuit showed sexualized and violent language from AI-generated characters.

Character.AI declined to comment on the litigation.

Common Sense Media reports that over 70% of US teenagers use AI technologies, raising concerns about exposure to harmful content.

Senator Richard Blumenthal and others introduced bipartisan legislation last week to restrict chatbot access for minors, requiring age verification and clear disclosures that users are interacting with non-human entities.

“AI companies are pushing treacherous chatbots at kids,” Blumenthal said. “Big Tech has betrayed any claim that we should trust companies to do the right thing on their own.”

Despite Character.AI’s new age restrictions, other platforms such as ChatGPT, Google Gemini, Grok by X, and Meta AI continue to permit underage use, according to their terms of service.

Jodi Halpern, an ethics expert from the University of California, compared unsupervised chatbot use to “letting your kid get in the car with somebody you don’t know.”