New research shows that machine learning (ML) can identify clinical notes written by physicians experiencing fatigue and provide insight into the quality of physicians' clinical decisions.
Published in Natural Communication,This study aimed to measure fatigue through clinical notes and to examine the effects of physician fatigue.
Researchers at the University of Chicago and the University of California, Berkeley collected doctor notes from Mass General Brigham from 2010 to 2012. Most of the notes collected were written on the same day of the patient's encounter.
For analysis, the researchers used data from 129,228 consecutive emergency department (ED) encounters. They identified a lead physician who wrote clinical notes for each visit, a total of 60 emergency physicians working over 11,592 shifts. The ED shift is “psychologically and physically demanding” for doctors, the researchers pointed out.
The researchers calculated the physician's workload by counting the number of days they were working over a seven-day rolling period that ended with the current shift. They defined “high workload” doctors as those who worked at least four days before their current shift (14.8%), compared to doctors who called “low workload” doctors with seven days (19%) current shifts. We then trained the ML model to identify notes written by high workload physicians.
This study shows that the model accurately identifies notes written by high-labor load physicians. We also identified notes written in situations related to high fatigue, such as overnight shifts and periods of high patient volume.
In particular, model identification signs of fatigue in clinical notes correlated with worse physician decisions. To assess correlations, researchers used previously developed ED quality measures. Whether to test patients with acute coronary syndrome (ACS)? They evaluated the “test yield” that determines the clinical value of the patient's test. High yields mean a high diagnostic rate for ACS, while low yields mean a risk of testing with clear patient benefits.
The researchers found that each standard deviation with an increased standard deviation of model identification fatigue results in a 19% lower yield for heart attack tests.
“The results show that the fatigue fatigue measurements used here are promising ways to measure and elucidate physician fatigue outcomes,” the researchers wrote.
Furthermore, the model found that clinical notes generated by large-scale language models (LLM) tend to correlate with fatigue more than those written by physicians. The researchers observed that the percentage of model identification fatigue in LLM-produced notes was 74% higher than that produced by physicians.
This indicates “the possibility that LLM introduces distortions into generated texts that are not yet fully understood,” the researchers said.
Anuja Vaidya has been covering the healthcare industry since 2012. Currently covers virtual healthcare situations such as Telehealth, Remot Patient Monitoring, and Digital Therapeutics.
