Study finds that flaws in AI response increase burden on doctors

Artificial intelligence is rapidly gaining popularity in the medical field, with the goal of streamlining important but tedious administrative tasks such as note-taking and graphing, allowing doctors and nurses to spend more time with patients.

But even if AI gives doctors more freedom to interact with patients, it may fall short because messages will be filled with errors and irrelevant details, according to new Dartmouth research presented at the Association for Computational Linguistics’ 2026 Annual Meeting and published in the conference proceedings.

As a result, doctors may spend more time editing responses than they do writing them, the researchers report.

“We believe that AI said Sarah Preum, assistant professor of computer science and co-corresponding author of the study with Parker Seegmiller, a graduate researcher at Preum’s PersistLab at Dartmouth.

Researchers conducted the first large-scale study of an online patient portal that uses AI to draft responses from doctors to patients. The team developed a tool that compares the AI-generated responses to a dataset of actual responses developed with medical experts at Dartmouth Health.

They then analyzed 146,000 conversations between 10,105 patients and their primary care physicians in a large rural health system. The study was approved by the Dartmouth Health System Review Board, and the team used the necessary methods to protect patient privacy, including anonymizing data when necessary.

The researchers also used tools to evaluate physician responses generated by Claude, Gemini, ChatGPT, and three smaller commercial platforms (Llama, Aloe, and Qwen).

It turns out that AI may sound like a doctor, but it can’t think like a doctor. ”

Sarah Preum, corresponding author and assistant professor of communication sciences

The researchers report that AI-generated answers often don’t match what clinicians actually write. This includes auto-responses that are too long, not asking follow-up questions, and using irrelevant or inaccurate medical details.

“While there are small-scale studies that say, ‘Oh, AI is amazing,’ we realized there was a gap in the existing literature on large-scale evaluations of this technology,” Preum says. “We not only wanted to measure the accuracy of the platform, but whether it actually helped the workload, which in this case was measured by how much editing doctors were doing.”

For example, the portal’s AI suggested telling a 32-year-old woman who was taking medication for acid reflux disease and was worried about constant nausea that she might need to adjust her diet because of the medication. Instead, the doctor asked if she could be pregnant.

Even small changes can add up to hundreds or thousands of messages, Preum says. “We don’t want physicians to spend their cognitive energy playing AI steward and fixing mistakes, just by integrating large language models into workflows and moving bottlenecks,” Preum says. “But if you’re not careful, that can happen.”

However, researchers have shown that by adapting AI to how individual doctors communicate, accuracy can be increased by 33% and edits reduced by 26%.

“If message generation is really efficient and high-quality and you’re asking for the right things, there’s a real potential to improve efficiency,” says co-author Tim Burdick, associate professor of community and family medicine at Dartmouth’s Geisel School of Medicine and family medicine physician at Dartmouth Health.

“We don’t foresee a time when a portal can respond to a patient without a clinician having to edit it first, but by refining the model, we will be able to address portal messages more quickly and with less mental energy,” Burdick says.

The study shows there is such a thing as “good” AI responses and provides a framework for implementing them into patient and physician portals, Preum said. These platforms are becoming increasingly common among large health systems and are often customized, she says.

“It took us a long time to figure this out, but if we’re going to measure how effective this technology is, we need to define what a good response is,” she says. “We can only improve things that can be measured and evaluated objectively.”

The researchers created a technique called TADPOLE (Thematic Agent Direct Priority Optimization for Learning Enhancement) that trains an AI platform using a hybrid model built from physician- and AI-generated responses.

They connected TADPOLE to six commercial LLMs and found that the drafted responses were better aligned with physicians’ standards for accuracy and quality of information. “This could potentially save busy clinicians one to two hours of work per day,” says Burdick.

Doctors and nurses are currently inundated with messages from patients and caregivers, who can write them online at any time. An ongoing project between Burdick and Preum Lab called PortalPal aims to use AI to streamline patient portals, including automating some steps when following up with patients for more information.

We are not yet at the point where we are removing clinicians from the workflow. ”

Tim Burdick, co-author, Associate Professor of Community and Family Medicine

Doctors at Burdick say AI-generated drafts save them about 25% of the time they spend writing short messages. “It’s easier to edit LLM-generated messages a little than to create them from scratch,” he says. However, longer drafts may contain incorrect or inaccurate information.

“If you need to edit 75% of your message, you may end up spending more time and energy making changes than if you were to create the message from scratch,” says Burdick. “I think we need to get to the point where physicians are editing less than 30% of the content to have a real impact.”

One benefit of AI’s redundancy is that it tends to be more empathetic and thorough than time-pressed doctors, the researchers found. For example, an AI is more likely to tell a patient who has an upset stomach, “I’m sorry to hear that you’re feeling nauseous.”

This means AI can be used to “encourage” doctors to show understanding and consideration for a patient’s situation, or to help patients feel more heard by answering patient questions more effectively, Preum said. The team developed example responses that showed empathy by praising patients for following their treatment plan (“You’re doing great with tapering off your medication”) and planning for changes in symptoms (“If you feel dizzy, call triage”).

The researchers also found that 65% of all portal messages examined came from people aged 55 and older, and patients aged 65 and older generated 24% of all messages. These numbers generally suggest that patient portals should be designed to accommodate older adults, Preum said.

Future research will explore how much time doctors actually spend editing automated drafts. The team also plans to evaluate the training model TADPOLE from a user perspective, studying whether and how it reduces physician workload and how physicians and patients rate its performance.

“This is one of the first studies to use real patient portal messages to establish a generative AI model. In that respect, it is innovative and shows that this is not a trivial task,” Burdick says. “We are not yet at the point where we are removing clinicians from the workflow.”

Burdick, Preum, and Seegmiller collaborated with co-author Joseph Gatto, who earned his Ph.D. from Dartmouth this year. Sarah Greer, former Dartmouth Health physician; and 2026 Dartmouth graduates Ganza Berrys Isingizwe and Rohan Ray.