The field of mental health treatment is undergoing an AI takeover with the rise of generative AI chatbots and solutions to assist mental health professionals, but new insights suggest that not everyone is ready to embrace the AI movement.
Recent data from news and intelligence agency Storyful shows that social media users are most concerned about the use of AI in mental health/therapy when discussing healthcare trends.
The data, which analyzed more than 49,000 posts across Reddit, X, Instagram, online forums, and news sites, found that AI in treatment accounted for 31.8 percent of all skeptical conversations about AI in healthcare.
While the therapeutics space is undoubtedly adjacent to the pharmaceutical industry, Tara Norton, senior vice president of intelligence at Storyful, pointed out that it’s worth the pharmaceutical industry paying attention to what’s happening in the therapeutics industry.
“This type of data and information is important for pharmaceutical industry communications personnel to guide and advocate for adoption by pharmaceutical companies and medical practices. Having this type of data and insight into where the fear is is extremely valuable,” he said, adding that the industry can learn from this data to inform its own communications around AI.
Skepticism about AI and treatment
Data from Storyful shows that user concerns about AI in therapeutics are widespread. The report noted that one Reddit user referred to AI-driven treatments as a “control system,” while others warned of “deadly risks” without trust.
Users said they were concerned about the loss of human connection in treatment, replacement jobs for mental health professionals, the risk of misdiagnosis if AI algorithms fail, and ethical concerns about how sensitive personal health data is stored and shared.
The report also highlighted that the lack of practical experience and expertise with AI is causing major concerns about its use in the therapeutic field.
“Because of the way LLM works, fear of misdiagnosis and ethics was pretty high. There was a concern that if the model was trained based on questions and dialogue, people might not be prompting the right way, and that could lead to inappropriate feedback for treatment. So there were concerns in that regard,” Norton said.
Privacy regarding medical data has also become a major concern, especially as healthcare and technology become more integrated.
“There has been some skepticism about big technology and the role of data protection in managing the execution of AI implementations,” Norton said.
Professional medical organizations are also weighing in, with the American Psychological Association raising concerns about unregulated therapeutic chatbots and calling on organizations like the Federal Trade Commission (FTC) to ensure protections for generative AI tools that can address mental health needs.
The American Medical Association recommends rebranding AI as “augmented intelligence” to emphasize that AI should be used alongside human interaction, rather than acting as a replacement.
Why this matters to the pharmaceutical industry
Norton said skepticism surrounding AI and treatments poses a communication dilemma for pharmaceutical companies, as the technology has the potential to close gaps in the care and access needs of Americans, especially at a time when health insurance premiums are at an all-time high.
Additionally, mental health influences the uptake and maintenance of medication. Research from the National Institutes of Health and MDPI shows that adherence to medications and related treatments is a factor that pharmaceutical companies need to consider when responding to individuals experiencing mental health problems.
On top of that, the AI industry will continue to grow. The healthcare AI market is expected to reach $187.7 billion by 2030, according to data from Storyful. As the industry grows, skepticism about it is also likely to grow unless action is taken.
If the industry wants to integrate and leverage AI, executives and communicators need to be mindful of the challenges of AI.
“From an industry perspective, the healthcare industry is in the best position to take full advantage of the use of AI. The healthcare industry is also more concerned about consumers and their health, so there is definitely a need for healthcare companies to have transparent safeguards in place to help people understand how their applications are being developed and implemented,” said Norton.
How should pharmaceutical companies respond?
Skepticism around AI has largely focused on the lack of trust and transparency in the tools, so pharma companies need to focus on how they communicate AI to prospects.
Norton noted that healthcare marketers need to be proactive when implementing AI and create messages that emphasize that AI will be used in conjunction with human review.
She outlined the differences between the two communication approaches, highlighting the following examples from the Storyful report:
- Wrong: “AI will improve healthcare through continuous surveillance.” This phrase reinforces fears of surveillance.
- Better: “AI speeds up daily tasks and care remains human.” This positions AI as a complement rather than a replacement.
He also emphasized that pharmaceutical companies need to be transparent about the processes and decisions they make to incorporate AI into their products.
“Pharmaceutical companies should make it a priority to share use cases where they have tried something that didn’t work, and in some cases learned from it, such as how it influenced further development that led to success,” she said.
“Highlighting use cases really helps people understand and trust your application of AI. Emphasizing ‘Why are we doing this? Are we in the business of saving lives or improving patient outcomes? What role does AI play in that?'” she added. “Leading with the benefits and reasons behind adoption goes a long way.”
