Machine learning-based tools have been applied to the diagnosis and management of various medical conditions. While these tools can often improve patient care and management, there are also concerns that this technology can become disconnected from the realities of front-line care. To address concerns regarding the application of this technology in the diagnosis and prognosis of multiple sclerosis (MS) patients, Tom AN Fuchs, MD, PhD, and colleagues conducted a retrospective, multicenter, case-control study of adult patients with relapsing-remitting MS. The research team collaborated with global partners through the MSBase research consortium to develop two complementary tools for individualized five-year risk estimation using machine learning. The first is DAAE-M (Donset period, aAt the onset of the disease, aGe, EExpanded Disability Status Scale and Illness –MIt is optimized for transparency, software-neutral use, and reduced adaptation bias. The second is ELIE (Empirical Landmark-Based Individualized Estimation of MS Progression Risk), which is optimized for landmark-based dynamic modeling, complex treatment history, and reduction of immortality time bias. The research results are neurology journal. Dr. Fuchs said: Doctor’s weekly magazine Regarding the survey results.
Physician’s weekly magazine: Can you describe the machine learning tool you developed to estimate the 5-year risk of progression for MS patients?
Dr. Fuchs: We recently developed the DAAE-M score, a machine learning-based tool to predict disease progression over five years in patients with multiple sclerosis. The DAAE-M score, which integrates information on disease status and use of disease-modifying therapies, provides accurate and consistent risk prediction across settings worldwide and can be used to estimate the risk of progression to secondary progressive MS or advanced disability that progresses independently of relapse (Rorsheider criteria).
Specifically, we built our tool using machine learning to improve accuracy, but we also took additional steps to ensure the final predictive model was transparent, interpretable, and usable in clinical practice within 30 seconds.
DAAE-M scores are available for free here: https://tomafuchs.com/daae-score/
And it will soon be available through the EPIC electronic health record global platform.
Why do you think it was important to add these tools to the ongoing diagnosis, treatment, and management of MS?
The DAAE-M score is perhaps most useful for engaging patients in the conversation. There is no one-size-fits-all treatment algorithm for people with multiple sclerosis. However, knowledge about the future can contribute to the dialogue between doctors and patients, especially during important opportunities for rehabilitation and treatment changes. We consider the DAAE-M score to be a free source of information that you can access if you want, and ignore it if you don’t. These tools provide information similar to weather forecasts. Some people bring umbrellas, while others choose to go out in T-shirts as the chances of rain are low. Reports themselves are another tool in your toolbelt and can be leveraged when useful.
What was the purpose of the research you developed?
The goal of our research was to develop a machine learning-based predictive model that is accurate, useful, and usable for physicians treating patients with multiple sclerosis. In this case, we focused on the consequences of progression to secondary progressive multiple sclerosis and progression to high-grade disability that progresses independently of relapse. These transitions are important for our patients because once they reach the progression of secondary progressive multiple sclerosis or high-grade disability, independent of relapse, their disability worsens more rapidly and responds less well to pharmacotherapy and rehabilitation.
What was the most impactful finding of your research?
I think the most impactful findings of this study are those related to the barriers between science and clinical practice. Machine learning is used and exploited at scale in scientific environments, but little attention is paid to global validation or user experience. We feel that practicing physicians are left out of too many conversations and that much of medicine is not meeting the needs of daily practice. We sought to address that social issue with this project, engaging with practitioners every step of the way to ensure we were delivering an experience that met their needs and expectations. This led us down an interesting path. There, we had to act not only as researchers but also as software developers, and we always had to aim for a specific user experience, rather than letting the technology itself determine the direction of our research.
How do you think these findings can be applied in practice?
We believe that tools similar to the DAAE-M score have the potential to impact daily clinical practice by facilitating a more nuanced, data-driven, risk-based dialogue between physicians and patients. That being said, we actually feel that the future of tools like the DAAE-M score is even more interesting. As physicians become accustomed to working with more accurate and easier-to-use predictive tools, practices evolve and require new tools to improve patient care. This interaction between use and need will generate future innovations. We want practicing physicians to feel empowered to contribute to that.
Why are machine learning tools so important for the progressive care of MS patients?
In neurology, we are accustomed to using tools such as CHA.2DS2– VASc score to assess stroke risk. This type of algorithm serves as a good example of how predictive tools can guide medical practice. But in this day and age, we can use machine learning to make these tools even better.
What still needs to be studied? What needs further research?
We identified important future directions for this research area and categorized them into two paths: depth and breadth. Depth Pathways will continue to validate the DAAE-M score across new countries and clinical settings to ensure its predictions are accurate, reliable, usable, and interpretable. One important feature of this study is that it is a type of “randomized controlled trial.” This study will investigate how machine learning-based tools, such as the DAAE-M score, impact clinical practice and patient outcomes. This is an important future direction for machine learning research. of width Research pathways are equally important. This pathway develops new predictive tools for new and important outcomes. In discussions with physicians, we identified the following outcomes that we believe are important to predict: higher-order disability (Expanded Disability Status Scale score of 6) and higher-grade cognitive impairment. We hope to cover more important outcomes with new predictive tools in the future, and we look forward to seeing how this area of research develops as we receive more input from physicians.
Is there anything else you would like to share?
We urge physicians to always encourage researchers to keep clinical practice in mind when conducting research. What are the barriers to clinical implementation? What tools do you wish you had? What prevents you from using them? What do you need to trust and use the tools on a daily basis? These barriers and ideas may be considered in the early stages of research planning, but it requires practicing physicians to have a seat at the table. Most researchers don’t have the lived experience that doctors have, so we need your voice.
