A simple adjustment to how AI assigns diagnostic codes can improve accuracy

New research from researchers at Mount Sinai Health System suggests that simple adjustments to how artificial intelligence (AI) assign diagnostic codes can significantly improve accuracy and even outperform performance physicians. Survey results reported in the online issue of September 25th かったんとかな [DOI: 10.1056/AIcs2401161]helps doctors spend less time on paperwork, reduce billing errors, and improve the quality of patient records.

Our previous research has shown that even the most advanced AI can sometimes generate meaningless code when left to guess. This time, we gave the model the opportunity to reflect and review similar past cases. Those small changes made a huge difference. ”

Eyal Klang, MD, senior author, co-responding senior author, Windreich Artificial Intelligence and Human Health Bureau Generation AI Chief of Icahn Medical School, Mount Sinai

US doctors spend hours each week. ICD Code – Assigning α-α strings and used to explain everything from the sprained ankle to a heart attack. However, large language models like ChatGpt often have trouble assigning these code correctly. To address this, researchers tried the “search before coding” method, which first encouraged AI to explain diagnosis in plain languages ​​and then encouraged them to select the most appropriate code from a list of real-world examples. This approach provided better accuracy, fewer mistakes, and performance equal or better than humans.

The team took advantage of 500 emergency department patient visits at Mount Sinai Health System Hospital. For each case, they fed doctor's notes to nine different AI models, including small open source systems. First, the model generated an early ICD diagnostic description. Using the search method, each explanation coincided with 10 similar ICD descriptions from a database of over 1 million hospital records and the frequency with which their diagnosis occurred. In the second step, the model used this obtained information to select the most accurate ICD description and code.

The emergency physician and two independent AI systems independently evaluated coding results without information about whether the code was generated by the AI ​​or clinician.

Overall, models using search steps often outweighed what the physicians were assigned to. Surprisingly, even small open source models worked well when they allowed them to “search” examples.

“This is about smarter support, not automation for automation's sake,” says co-corresponding senior author Girish N. Nadkarni, MD, MPH, Chair of the Windreich Department of Artificial Intelligence and Human Health, Director of the Hasso Plattner Institute for Digital Health, and Irene and Dr. Arthur M. Fishberg Professor of Medicine at the Icahn School of Medicine at Mount Sinai, and Chief AI Officer for the Mount Sinai Health System. “If doctors can reduce the time they spend coding, reduce billing errors, and improve the quality of their data with an all-around, transparent system, that's a huge win for patients and providers too.”

The authors emphasize that this search-enhanced method is designed to support, rather than human surveillance, rather than alternatives. Although the claim has not yet been approved and was specifically tested with primary diagnostic codes from emergency visits discharged from emergency visits, it encourages the possibility of clinical use. Researchers have seen immediate uses, such as suggesting codes for electronic records and errors that flag them before they are requested.

Investigators are currently integrating the method into Mount Sinai's electronic health record system for pilot testing. They hope to extend it to other clinical settings and include secondary and procedural codes in future versions.

“The whole picture here is the possibility of AI that changes the way patients are cared for. When technology reduces the administrative burden of doctors and other providers, there is time for direct patient care. It is good for clinicians and is suitable for health systems of all sizes.” “Using AI in this way improves the ability to provide careful and compassionate care by spending more time with patients. This strengthens the foundation of hospitals and health systems everywhere.”

The title of this paper is “Evaluation of a large-scale searched language model for medical coding.”

The authors of the studies described in the journal are Eyal Klang, Idit Tessler, Donald U. Apakama, Ethan Abbott, Benjamin S Glicksberg, Arnold Monique, Akini Moses, Ankit Sakhuja, Ali Soroush, Alexander W. Charney, and David L. Girish N Nadkarni.

This work was supported by the National Center for Translation Science Clinical and Translational Sciences (CTSA) Grant UL1TR004419. The studies reported in this publication are also supported by the National Institutes of Health's Research Infrastructure Office under Grant Awards Nos. S10OD026880 and S10OD030463.