1. A new artificial intelligence (AI) system, AI-MARRVEL (AIM), was trained using samples diagnosed by experts from the National Board of Medical Genetics and Genomics. The system was then tested on three patient data sets.
2. Compared to the four existing top-performing algorithms, AIM doubled the number of resolved cases. AIM achieved an accuracy rate of 98% and identified 57% of diagnosable cases.
Level of evidence: 2 (good)
Research Summary: Mendelian diseases are caused by one or a few mutations in one gene, but identifying these mutations is time-consuming and requires a broad knowledge base. A cost-effective solution is bioinformatics genetic analysis, but existing tools have limited accuracy and were developed using simulated data. Mao and colleagues used 3.5 million mutation data points to develop a new AI system, AIM, which was designed using genetic principles and expert clinical expertise. AIM's accuracy was then tested against four existing best-performing bioinformatics tools using data from three different patient groups. The study found that AIM outperformed all four comparison tools on all three datasets. As the amount of training samples increased, AIM's accuracy improved from 54% to 66% after incorporating additional engineering features. However, like other tools, AIM performed worse in cases of recessive inheritance patterns compared to those with dominant inheritance patterns. AIM was then further modified to create a model dedicated to diagnosing recessive cases, achieving an accuracy of 63.4%. Similarly, additional training and filters improved performance in diagnosing heterozygous mutations. Overall, this study demonstrated the potential of AIM in identifying genetic mutations for diagnosing Mendelian diseases and its superiority over existing algorithms.
Click here to read the NEJM AI study
Click to read the accompanying editorial in NEJM AI
Related article: Open-source artificial intelligence system helps diagnose severe Mendelian disease in infants
detail [cross-sectional study]: Mao and colleagues developed AIM using 3.5 million mutation data points from samples diagnosed and selected by experts certified by the American Board of Clinical Genetics and Genomics. AIM was designed based on relevant knowledge such as minor allele frequency, mutation effects, inheritance patterns, phenotype concordance, and genetic constraints, as well as various aspects of genetic diagnostic technologies. The authors compared AIM's performance with four top-performing benchmark algorithms (Exomiser, LIRICAL, PhenIX, and Xrare) on three independent data sets (totaling 1377 patients). Throughout the study, AIM was modified and additional models were created to evaluate changes in performance in specific diagnostic contexts (e.g., diagnosing recessive disorders).
AIM achieved higher accuracy than its four comparators in all three datasets, diagnosing 57% of the diagnosable cases out of 871 cases. For comparison, the current diagnostic rate for such diseases is 30-40%. With additional engineering features, AIM's accuracy increased from 54% to 66%, indicating AIM's ability to capture underlying patterns within additional training data. As with other algorithms, AIM performed poorly in recessive cases, so AIM-Recessive was developed, achieving an accuracy of 63.4%. Finally, new AIM models not connected to established disease databases may be able to identify new disease genes and variants with limited patient data. Despite these successes, AIM has been trained primarily on cases with coding variants, and its ability to analyze non-coding variants is unknown.
In conclusion, the authors evaluated the ability of AIM to diagnose Mendelian diseases by analyzing genetic mutations. The superiority of AIM over existing algorithms demonstrated its potential to be a more cost-effective way to interpret genetic mutations and improve patient outcomes.
Image: PD
©2024 2 Minute Medicine, Inc. All rights reserved. No work may be reproduced without the express written consent of 2 Minute Medicine, Inc. Please contact us here for licensing inquiries. No article should be construed as medical advice, and no such intent is held by the author or 2 Minute Medicine, Inc.
