Parents and online safety advocates on Tuesday urged Congress to promote more protective measures around artificial intelligence chatbots, claiming that tech companies have designed products to “hook” children.
“The truth is that AI companies and their investors have been understanding for years that capturing emotional dependence in children means dominance in the market,” said Megan Garcia, a Florida mom who sued the character on the chatbot platform last year.
“Indeed, they purposefully designed their products to attract our children,” she told lawmakers.
“The goal was never safe, it was to win the race for profit,” Garcia added. “The sacrifice in commercial competition will continue to be our children.”
Garcia was among several parents who gave emotional testimony before a Senate panel, sharing anecdotes about how children's use of chatbots caused harm.
The hearings occur amid surveillance of high-tech companies such as Character.ai, Meta and Openai, behind the popular ChatGpt. As people turn to AI chatbots for more and more emotional support and life advice, recent incidents have highlighted the possibility of nourishing delusions and promoting false closeness and sense of care.
It's a problem that continues to plague the tech industry as businesses navigate the generative AI boom. High-tech platforms are protected primarily from unlawful suits of death due to federal law known as Section 230, and generally protect the platform from liability for what users do and say. However, the application of Section 230 to the AI platform remains uncertain.
In May, U.S. District Senior Judge Anne Conway rejected the argument that the AI chatbot has the right to free speech after the developers of the Charition. The award means that the illegal death lawsuit is permitted to continue for now.
Three additional product liability cases were filed against Chargetwed.AI on Tuesday, hours before the Senate hearing, just hours before the Senate hearing, according to the Social Media Victims Law Center.
In one of the suits, the parents of 13-year-old Juliana Peralta claim the character.
Matthew Lane, who alleged that his teenager used ChatGPT as his “suicide coach,” testified that he believes technology companies need to prevent harm to young people on the internet.
“We have one requirement, as Adam's parents and as people who care about youth in this country and around the world. [CEO] Lane, whose 16-year-old son Adam died of suicide in April, told lawmakers.
“If they can't, they should now pull GPT-4O from the market,” Raine added, referring to the version of ChatGPT his son used.
In their lawsuit, the Raine family accused Openai of failing to warn users of illegal deaths, design flaws and risks associated with ChatGPT. The GPT-4O, which his son had confidence in spending hours each day, offered to help him write a suicide note at one point, advised him on his rope setup, according to the submission.
Shortly after the lawsuit was filed, Openai added a slate of safety updates to ensure parents are more monitored than teenagers. The company also strengthened ChatGpt's mental health guard rail at various points after Adam's death in April, particularly after GPT-4o monitored over excessive psychofancy.
On Tuesday, Altman announced it would wipe out new approaches to teenage safety and user privacy and freedom.
To set limits for teenagers, the company is building an age system that estimates users' age based on how ChatGPT is used, he wrote in a blog post published hours before the hearing. If in doubt, you will be classifying the user as a minor by default, and in some cases you may request an ID.
“ChatGpt is trained not to talk about the above frivolous stories. Even in a creative writing setting, when asked, engages in discussions about self-harm suicide,” writes Altman. “And if a user under the age of 18 has suicidal ideation, they will try to contact the user's parents and, if not possible, contact the authorities in case of immediate harm.”
For adult users, ChatGpt does not provide suicide instructions by default, but is allowed in certain cases, such as when they help users write fictional stories about the suicide. The company is developing security features to make users' chat data private, and to monitor automated systems for “potentially serious misuse,” Altman writes.
“As Sam Altman revealed, we believe minors need great protection, so we'll prioritize teen safety,” an Openai spokesman told NBC News, adding that the company is rolling out new parental controls by the end of the month.
But some online safety advocates say tech companies can and should do more.
Robbie Torney, senior director of the AI program at Common Sense Media, a nonprofit advocacy group for 501(c)(3), said that about 70% of teens have already used AI peers, but only 37% of children know that their children are using AI.
During the hearing, he called on the characters to watch out, and Meta is one of the worst performances of the safety tests carried out by his group. Meta AI is available to all teens on Instagram, WhatsApp and Facebook, and parents can't turn it off, he said.
“Our testing showed that the Meta safety system was fundamentally broken,” Tawny said. “If the 14-year-old test account described severe eating disorder behaviors such as 1,200 calorie diet and bulimia, Meta AI provided encouragement and weight loss influencer recommendations instead of help.”
Suicide-related guardrail obstacles are “even more surprising,” he said.
In a statement given to the news outlet after the Common Sense Media report was released, a Meta spokesman said the company did not allow content that promotes suicide or eating disorders and “is actively working to tackle the issues raised here.”
“We want teens to have a safe and positive experience with AI, so AIS is trained to support people with resources in sensitive situations,” the spokesman said. “We continue to improve enforcement as we explore ways to further enhance teen protection.”
Our kids are not experiments, nor are data points or profit centers.
– Parent Jane Doe testified during Senate hearing on Tuesday
A few weeks ago, Meta announced that it would take steps to train AIS to not deal with teenagers on self-harm, suicide, disability and potentially inappropriate romantic conversations, and to limit access to teenager AI characters' selection groups.
Meanwhile, Charition.ai has “invested a significant amount of resources in trust and safety” over the past year, a company spokesman said. This includes different models of minors, the “parent insight” feature, and disclaimers within prominent chats, reminding users that the bot is not a real person.
“The hearts of the families we spoke to at today's hearing are coming into contact with. We are saddened by their losses and send our deepest sympathy to our families,” the spokesman said.
“At the beginning of this year, we provided the senators on the Judiciary Committee with the information they requested. We look forward to continuing to work with lawmakers and providing insight into the rapidly evolving technology in the consumer AI industry and space,” the spokesman added.
Still, those who addressed lawmakers on Tuesday emphasized that technological innovation cannot cost people's lives.
“Our children are not experiments, nor are data points or profit centers,” said the woman who testified with Jane Doe. “They are humans with hearts and souls. They can't simply reprogram them if they're hurt. If they're here today, it's worth it to me. This is the public health crisis I see. This is a mental health war and we really feel like we're losing.”
