California lawmakers hope Gov. Gavin Newsom will approve a bill they have passed that aimed to make artificial intelligence chatbots safer. But as the governor considers whether to sign the law, he faces a familiar hurdle. It's a dispute from high-tech companies that say new restrictions will hinder innovation.
California companies are world leaders of AI and have spent hundreds of billions of dollars ahead of the race to create the most powerful chatbots. The rapid pace is worrying for parents, and lawmakers are worried that chatbots are hurting their children's mental health by putting them at self-harm and other risks.
Parents claiming chatbots encouraged teens to do harm to themselves before dying from suicide. They also want more guardrails.
It has echoed across the country's capital and various states in search of more AI regulations. Despite the Trump administration's “AI Action Plan” suggests cutting the deficit to encourage AI development, lawmakers and regulators on both parties are addressing child safety concerns surrounding chatbots that answer questions and act as digital companions.
This month, California lawmakers passed two AI chatbot safety invoices lobbyed by the tech industry. Newsom must approve or reject them by mid-October.
The decision to the High Stakes puts the governor in a tricky place. Politicians and tech companies want to assure the public that they are protecting young people. At the same time, tech companies are trying to expand the use of chatbots in their classrooms, opposed new restrictions they say are going too far.
Suicide Prevention and Crisis Counseling Resources
If you or someone you know is struggling with the idea of suicide, call 9-8-8 for help from a professional. The US's first national triple-digit mental health crisis hotline 988 connects callers with trained mental health counselors. “Home” text to US and Canada 741741 to reach the text line of crisis.
Meanwhile, if Newsom runs for president in 2028, he may need more financial support from wealthy tech entrepreneurs. On September 22, Newsom promoted state partnerships with tech companies on AI initiatives, touting how the tech industry has promoted the California economy, calling the state “the epicenter of American innovation.”
He has rejected AI safety laws in the past, invoice Last year it split Silicon Valley's tech industry because it thought the governor had given the public “false safety.” But he also shows he is trying to balance his safety concerns with ensuring that California tech companies continue to take control of AI.
“We support risk taking because we have a sense of responsibility and accountability, but we are not reckless,” Newsmom said in a discussion with former President Clinton at the Clinton Global Initiative event on Wednesday.
Two bills sent to the governor — Congressional Bill 1064 and Senate Bill 243 — aim to make AI chatbots safer, but face harsh opposition from the tech industry. It is unclear whether the governor will sign both bills. His office declined to comment.
AB 1064 prohibits individuals, businesses and other entities from making companion chatbots available to California residents under the age of 18, unless the chatbot is “nearly incompetent” to harmful behavior, such as encouraging chatbots to engage in diets with self-harm, violence or disabilities.
SB 243 requires the operator of a companion chatbot to notify certain users that the virtual assistant is not human.
Under the bill, chatbot operators must have procedures to prevent the production of suicide or self-harm content and include guardrails such as introducing users to suicide hotlines and crisis text lines.
They must inform minor users at least every 3 hours that they will take a break, and the chatbot is required to be non-human. Operators should also implement “rational measures” to prevent companion chatbots from generating sexually explicit content.
Members of the group, including Openai, Meta, Google and others, “agree with the intentions of the bill,” but continue to oppose them, said Technet, a high-tech bee group that said in a statement.
AB 1064 “imposes ambiguous and ineffective restrictions while reducing students from valuable AI learning tools. “SB 243 establishes clearer rules without blocking access, but we are concerned about its approach.”
A Meta spokesman said the company “has concerns about unintended consequences like AB 1064.” The tech company has launched a new Super PAC to combat AI regulations that it considers too ambitious, and is seeking more parental control over how children use AI, Axios reported Tuesday.
An opponent led by ASSN in the computer and communications industry. He said he actively lobbyed against AB 1064, which threatened innovation, threatened businesses that face more lawsuits and were at a disadvantage against California businesses that had to determine whether they wanted to continue operating in the state.
Advocacy groups, including Common Sense Media, a nonprofit sponsoring AB 1064 and recommending that minors should not use fellow AI, are urging newsom to sign the bill to the law. Atty, California. General Rob Bonta also supports the bill.
The Electronic Frontier Foundation said the SB 243 is too broad and will encounter the issue of free speech.
Several groups, including Common Sense Media and Tech Surveillance California, have removed support for SB 243 after changes were made to the bill. Changes are restricted, including exemptions for certain chatbots for virtual assistants that receive certain notifications and are used in video games and smart speakers.
Lawmakers who introduced the Chatbot Safety Act want the governor to sign both bills, claiming that they can “work in harmony.”
Sen. Steve Padilla (D-Chula Vista), who introduced SB 243, said he believes that new rules will make AI safer, even with changes.
“We have technology that can be good, incredibly powerful, but incredibly rapidly evolving. We can't miss the windows that offer common sense guardrails here to protect people,” he said. “I'm happy with where the bill is.”
Council member Rebecca Bauer-Kahan (D-Orinda), who co-authored AB 1064, said her bill leaves a balance of AI benefits while protecting against danger.
“When kids are involved in chatbots, they want to make sure that they are not creating unhealthy emotional attachments.
During the legislative meeting, lawmakers heard from grieving parents who lost their children. AB 1064 highlights two well-known lawsuits. One is another lawsuit against San Francisco ChatGpt maker Openai and Character Technology, the developer of Chatbot Platform Character.ai.
Character.ai It is a platform where people can create and interact with digital characters that mimic real and fictional people. Last year, Florida mom Megan Garcia alleged in a federal lawsuit that she accused her of not providing help when she notified her or expressed her thoughts of suicide to a virtual character that she was injuring her to her son, Selwell Setzer III.
More families sued the company this year. a Character.ai The spokesman was very deeply concerned with user safety, saying that “encouraging lawmakers to properly create laws that promote user safety, ensuring that there is ample space for innovation and free expression.”
In August, Adam Lane's California parents sued Openai, claiming that ChatGpt provided teenage information on how to commit suicide.
Openai said it plans to release enhanced safeguards and parental controls. Its CEO Sam Altman wrote in a blog post in September that the company believes minors need “significant protection” and that the company prioritizes “teen privacy and freedom ahead of itself.” The company declined to comment on the California AI chatbot invoice.
For California lawmakers, the clock is ticking.
“We're doing our best,” Bauer Kahan said. “The fact that we have already seen our kids lose their lives to AI tells me that we're not moving fast enough.”
