iStock.com/nuttapong punna
A pilot study evaluating an AI-powered documentation tool found that physicians incorporated machine-generated discharge summaries into more than half of patient records, with limited safety concerns and measurable changes in clinician experience. The result is JAMA network open.
This study was conducted in the inpatient medicine department of a university health system and evaluated an agent-based workflow that uses a large-scale language model to create hospital course summaries from clinical notes. Over a 10-week period, the system generated 1,274 summaries for 384 discharges.
Deploying AI documentation tools and safety risks
Physicians used AI-generated content in 57% of discharge summaries. In cases where clinicians provided feedback, most drafts were rated as unlikely to cause harm. Only one summary was flagged as potentially causing moderate harm. According to the study’s authors, “the reviewing physician noted that the summary did not mention that a full 9-day course was required.” [of oral antibiotics] It has already been completed and the indication is [to transition to oral antibiotics] It was preventive due to an unrelated illness. ” A summary ruling determined there was no risk to this, but it was flagged to ensure a conservative risk assessment. There were no cases with serious risks.
The most common problems concerned missing information or partial description of clinical details. There were fewer inaccurate statements and fabricated content was rare. The physician identified and corrected these issues during a routine examination prior to finalizing the documentation.
The system automatically created summaries from existing records, such as admission notes and daily progress reports, and provided clinicians with a draft for use as needed. This approach allowed integration into existing workflows without changing documentation responsibilities.
Physician burden and impact on workflow efficiency
This study also investigated the impact of AI-assisted documentation on clinician workload, measured using EHR activity data and survey-based measures. Participating physicians’ burnout scores decreased from 1.75 to 1.20 (P=0.03) below the threshold for work-related fatigue on the Stanford Vocational Fulfillment Index Job Fatigue Scale.
Documentation time changes were less consistent. Although the majority of physicians (71.4%) recorded a reduction in discharge summary completion time, the overall reduction was ~2.9 minutes and was not statistically significant (P=0.13). However, clinicians reported that they realized time savings in 67% of cases when using AI-generated drafts. In 32% of responses, clinicians estimated they saved more than 15 minutes per summary.
These results are consistent with previous studies. According to the study’s authors, “The main benefit of generative AI tools lies in cognitive offloading rather than time efficiency. This explains why the value proposition shifts from efficiency to sustainability and when burnout improves.” [and] When the clock doesn’t match the time. ”
Impact of AI clinical tools on healthcare system implementation
The findings provide initial data for the future use of AI documentation tools in real-world clinical settings. Unlike previous retrospective evaluations, this study measured performance during active patient care.
For healthcare organizations, discharge summaries represent important documentation requirements related to care transitions, quality metrics, and reimbursement workflows. Tools that help generate these summaries can impact both operational efficiency and employee experience.
This study was conducted in a single unit with a limited group of physicians, so results may vary across settings. The researchers noted that further evaluation is needed to assess long-term performance, scalability, and integration with broader clinical systems.
