A year ago, Illinois Sen. Bob Morgan heard from a group of social workers. They asked him to look into the artificial intelligence therapy bot.
Morgan said he heard of individuals dealing with substance abuse, mental illness, suicidal ideation and other life and death issues, “new examples of AI therapy bots that actually provide bad advice, and sometimes dangerous advice.”
In a particular example of a therapist, Morgan said the chatbot told addicted people to take more drugs “as it was good for now.”
That's why state representatives have begun drafting bills that would prohibit therapists from using AI for non-administrative purposes, such as note-taking and scheduling. The law also states that chatbots cannot diagnose or treat mental illnesses and cannot sell themselves as if they were.
“We're an AI bot pretending to be a therapist, that's inappropriate and we're going to shut it down,” Morgan said.
Illinois is not the first state to pass laws relating to the use of AI in psychotherapy. Utah and Nevada passed laws this year to curb the use of chatbots and other tools in mental health treatment.
This “patchwork approach” by the state could continue, according to Beyerright, senior director of healthcare innovation at the American Psychological Association.
Wright said, “Because there is uniformity and greater specificity in various conditions that can produce better results.
But such restrictions are unlikely given the state's House of Representatives' efforts this summer to regulate artificial intelligence for a decade as part of President Donald Trump's one big beautiful bill act. The US Senate ultimately voted to strike that provision from the bill.
Federal law aside, Wright said the ban on AI therapy like in Illinois doesn't address one of the biggest issues plaguing her field.
“They call themselves peers. They say they help you in your loneliness. But when you read the fine print, they say very clearly, “We're not medical assistance,” Wright said.
She added that the business model for these platforms is to keep visitors up through validation and enhancement.
“Essentially, they tell you exactly what you want to hear, and this is an antithesis of treatment,” she said.
But Wright sees the future in which mental health chatbots are “rooted in psychological science, rigorously tested and co-created with experts, and there are people who monitor interactions.”
Such tools are possible if federally regulated – Wright added, help fill the void in the growing mental health crisis in the US.
Until then, licensed psychologists like Michelle Carnacy Powell say they will look and wait.
“I'm skeptical of AI,” she said, adding that the Illinois ban is a good starting point, but that may not be enough.
Kalnasy Powell currently uses AI for billing, but not to take notes during sessions with patients. Some of her peers use dictation software that complies with patient confidentiality laws.
“Even so, I'll ask you the questions when I read the terms of use,” she said. “It's like sending a session to ether. What are you doing with that content? Is it really being removed?”
She added that her work is very personal and vulnerable.
“It's a privilege and an honor to be able to hear the stories of people, both joy, happiness and sadness,” she said. “[If] It somehow gets caught up in it in a way none of us can predict, but it will hurt our clients.
