Study Finds AI Chatbots Provide High-Quality, Empathetic Answers to Patient Questions

Research published in JAMA Internal Medicine Artificial intelligence assistant-generated responses to patient questions are shown to outperform physician responses in terms of quality and empathy.

Study: Compare responses of doctors and artificially intelligent chatbots to patient questions posted on public social media forums. Image credit: TierneyMJ / Shutterstock

Background

Due to social restrictions, virtual health systems have grown significantly during the 2019 coronavirus disease (COVID-19) pandemic. This resulted in a 1.6x increase in electronic messages to patients, and a corresponding increase in workload and stress for healthcare workers. All these factors collectively can lead to a situation where most patient messages are ignored or not answered satisfactorily.

Current strategies to reduce the burden of virtual care include limiting electronic message notifications, charging for responses, or delegating messages to untrained medical staff. However, these strategies limit patient access to quality medical support. Healthcare systems are now looking to deploy artificial intelligence (AI) assistants to reduce the workload of healthcare workers.

In the current study, scientists investigated the ability of an AI chatbot assistant (ChatGPT) to provide high-quality, empathetic responses to patient medical messages. In particular, we compared the chatbot’s responses with physicians’ responses to questions asked by patients on social media platforms.

ChatGPT is a new generation AI technology driven by large scale language models. This chatbot is widely known for its ability to produce human-like quality text on a wide range of topics.

key findings

This study utilized a public database of questions from public social media platforms to randomly select 195 interactions containing unique patient questions and unique physician answers.

A comparison of the doctor’s responses and the chatbot’s responses revealed that the average length of the doctor’s responses was significantly shorter than the chatbot’s responses. Of the selected interactions, about 94% consisted of only her one patient question and her one physician answer. The rest of the exchange consisted of the doctor’s two separate responses to a single patient’s question.

Raters (a team of qualified medical professionals) who analyzed the selected exchanges preferred chatbot responses over physician responses in 78% of the 585 ratings.

According to their report, chatbot responses are of much higher quality than doctor responses. They used a Likert scale to sort the responses into five groups. very bad, bad, acceptable, good, or very good. The results revealed that the chatbot’s responses were better than good, and the doctor’s responses were acceptable.

Prevalence of responses rated below acceptable For doctors, the quality was 10 times higher. In contrast, the prevalence of assessed responses was good again very good 3x for chatbots.

Raters rated the chatbot’s response to be much more empathetic than the doctor’s response. They found that the doctor’s response was 41% less empathetic than the chatbot’s response. In addition, the prevalence of responses rated I can sympathize a little It was five times higher for doctors. In contrast, the prevalence of assessed responses was sympathetic again very sympathetic 9x for chatbots.

Significance of research

In this study, we found that AI chatbot assistant-generated responses to patient medical messages were superior to physician-provided responses in terms of quality and empathy. Based on these findings, the scientists recommend that his AI chatbot assistant can be introduced into electronic messaging clinical settings. However, messages generated by chatbots must be reviewed and edited by a physician to improve accuracy levels and limit potential false or fabricated information.

The high-quality, empathetic responses generated by chatbots can help respond quickly to patient medical questions. This is necessary to reduce unnecessary clinic visits and free up resources for more valuable patients. In addition, these responses may improve patient outcomes by increasing treatment adherence and compliance and reducing the frequency of missed appointments.

As the scientists also noted, the study primarily assessed the quality of responses generated by chatbots. However, this study did not evaluate how AI assistants could enhance clinicians’ responses to patient questions.

usechatgpt initialization successful

Reference magazines:

usechatgpt initialization successful

written by

Dr. Sanchali Sinha Dutta

Dr. Sanchari Sinha Dutta is a science communicator who believes in spreading the power of science to every corner of the world. She holds her Bachelor of Science (B.Sc.) and Master of Science (M.Sc.) degrees in Biology and Human Physiology. After receiving her master’s degree, Sanchari continued her PhD studies. in human physiology. She has authored over ten of her original research papers, all of which have been published in world-renowned international journals.