Artificial intelligence chatbots do not judge. Tell them the most private and vulnerable details of your life, and most of them may validate you and even provide advice. This has led many people to turn to applications such as Openai's ChatGpt for Life guidance.
However, AI's “therapy” carries significant risks. In late July, Openai CEO Sam Altman warned ChatGpt users against using chatbots as “therapists” due to privacy concerns. The American Psychological Association (APA) asked the Federal Trade Commission to investigate “deceitful practices” that AI chatbot companies claim to be “used by handing over themselves by handing them as trained mental health providers.”
“What stands out to me is how it sounds,” says C. Vaile Wright, a licensed psychologist and senior director of the APA's Department of Healthcare Innovation, which focuses on the safe and effective use of technology in mental health care. “Even six or 12 months ago, the level of technology refinement is pretty incredible.
Supporting science journalism
If you enjoy this article, consider supporting award-winning journalism. Subscribe. Purchase a subscription helps ensure a future of impactful stories about discoveries and ideas that will shape our world today.
Scientific American We talked with Light about how the AI chatbots used in treatments can be dangerous and whether it is possible to design something safe and secure that will definitely help.
[An edited transcript of the interview follows.]
What have you seen in AI in the world of mental health care over the past few years?
I think I've seen two big trends like that. One is AI products aimed at providers, and these are primarily management tools to help you with your treatment notes and claims.
Other major trends are [people seeking help from] Chatbot directly from consumers. And not all chatbots are the same, right? There are several chatbots that have been developed specifically to provide emotional support to individuals. That's how they sell. Next is the offer of these more general list chatbots [such as ChatGPT] It is not designed for mental health purposes, but it is known to be used for that purpose.
What concerns do you have about this trend?
When individuals use chatbots, we have a lot of concerns [as if they were a therapist]. These aren't just designed to deal with mental health and emotional support. Because that's a business model, they're actually coded to keep you on the platform as long as possible. And the way they do it is to unconditionally verify and strengthen it, up to the point of almost homosexuality.
The problem with that is that if you are the one who comes to these chatbots for help and you express harmful or unhealthy thoughts and behaviors, the chatbot only strengthens you to continue doing so. on the other hand, [as] Therapist may be examining, but it is my job to point out when you are engaged in unhealthy or harmful thoughts and behaviors and help you deal with that pattern by changing it.
What's even more annoying is when these chatbots actually call themselves therapists or psychologists. They sound very convincing and legal, which is pretty scary. Of course, it's when it's not.
Some of these apps are explicitly sold as “AI therapies” despite not being licensed therapy providers. Are they allowed to do that?
Many of these apps actually work in a gray space. The rule is that if you make the claim to treat or treat any kind of mental disorder or mental illness, it should be regulated by the FDA. [the U.S. Food and Drug Administration]. But many of these apps will do that [essentially] In their fine print, “We do not treat or provide interventions [for mental health conditions]. ”
They are marketing themselves as a consumer wellness app, so they don't fall under FDA surveillance. [where they’d have to] It shows at least a minimum level of safety and efficacy. You are not responsible for doing these wellness apps either.
What are the main privacy risks?
These chatbots have no legal obligation to protect any information at all. So it's not just possible [your chat logs] You'll be summoned, but in the case of a data breach, do you really want these chats in a chatbot that everyone can use? For example, would you like your boss to know that you are talking to a chatbot about alcohol use? I don't think people know they put themselves at risk by putting them [their information] There.
Here are the differences between therapists: Of course I might get subpoenaed, but I have to work under HIPAA [Health Insurance Portability and Accountability Act] Legal and other types of confidentiality laws as part of my ethics code.
You said that some people may be more vulnerable to harm than others. Who is at the most risk?
Certainly young people, such as teenagers and children. That's because they are developmentally less mature than older people. When something doesn't feel right, they may not be likely to trust their gut. And not only is it more comfortable for young people with these technologies, there is some data. They say they trust them more than people because they actually feel that they are not judged by them. I also think anyone who is emotionally or physically isolated or has existing mental health challenges is certainly at a great risk.
What do you think is driving more people to seek help from chatbots?
I think it's very human to want to find answers to what bothers us. In a way, chatbots are just the next iteration of the tools to do that. Before that was Google and the Internet. Before that, it was a self-help book. However, accessing mental health care for a variety of reasons has been complicated by the fact that a system is broken, which is extremely difficult. This is because there is a shortage of providers. I also hear that your provider is preventing you from purchasing insurance. This reduces access. Technology must play a role in helping to address access to care. You need to make sure it's safe, effective and responsible.
What is a safe and responsible way?
If there's no company doing it yourself, that's probably not likely, but certainly we've made some changes.[the APA’s] Priority is law at the federal level. That regulations include protecting confidential personal information, several restrictions on advertising, minimizing addictive coding tactics, and specific audit and disclosure requirements. For example, businesses may need to report the number of times suicidal ideations detected and known attempts or completions. And certainly, companies cannot call chatbots psychologists or therapists, as we want laws that prevent misrepresentation of psychological services.
How can an ideal and safe version of this technology help people?
Let's say the two most common use cases I'm thinking are one in the morning and you're on the crisis of a panic attack. Even if you are in treatment, you cannot reach your therapist. So, what if there was a chatbot that would help you calm you down and remind you of tools that would help you regulate your panic before the panic gets worse?
Another use we often hear is to use chatbots as a way to practice social skills, especially for young people. So you want to get closer to new friends at school, but you don't know what to say. Can I practice with this chatbot? Then ideally do that practice and use it in real life.
There seems to be tension in trying to build a safe chatbot to provide mental help to someone. The less flexible and scripted scripting, the more control over the output and the higher the risk of saying something that causes harm.
I agree. I think there's definitely tension there. I think it's part of what's being made [AI] The choice for people to deal with mental health through chatbot well-developed wellness apps is that they are very attractive. They really feel like this interactive interaction, but some of the engagement in these other apps is often very low. Most people who download [mental health apps] Use them once and abandon them. You'll see much more engagement [with AI chatbots such as ChatGPT].
We look forward to the future of having a mental health chatbot rooted in psychological science and rigorously tested mental health chatbot. Co-created with experts. It is ideally regulated by the FDA as it is built to address mental health. For example, there is a chatbot called Therabot, developed by researchers at Dartmouth. [College]. I'm not currently in the commercial market, but I think there's a future within it.
