Improved reliability of AI for medical diagnosis

Two studies, led by researchers at the Johns Hopkinskinmel Cancer Center, Ludwig Center and the Johns Hopkins Whiting School of Engineering, report on powerful new ways to significantly improve the reliability and accuracy of artificial intelligence (AI) in many applications. As an example, they apply a new method to early cancer detection from blood samples known as liquid biopsies.

One study reports on the development of a generalized hypothesis test (informed by MIGHT), an AI method that researchers created to meet the high level of trust needed for AI tools used in clinical decision-making. To illustrate the benefits of Might, they used it to develop a test for early cancer detection using non-circulating cellular DNA (CCFDNA). Companion studies also found that fragmentation patterns of CCFDNA used to detect cancer also appear in patients with autoimmune and vascular disease. It was expanded to incorporate data from autoimmune and vascular diseases obtained from Johns Hopkins and other institutions that treat and study these diseases to develop tests that have high sensitivity to cancer but reduce false-positive results.

The study, supported in part by the National Institutes of Health, was published in the minutes of the National Academy of Sciences on August 20th.

Written by three Johns Hopkins researchers, Dr. Ed Catmal, co-founder of Pixar, and Microsoft's Chief Data Scientist at AI at Good Lab Juan La Vista Ferez, the related article was published simultaneously in Cancer Discovery, a publication by the American Association for Cancer Research. We will discuss the challenges of incorporating AI into clinical practice.

You can use real data to fine-tune itself, use tens of thousands of decision trees to check its accuracy on different subsets of data, and apply it to any field using big data, ranging from astronomy to zoology. It is particularly effective in analyzing biomedical datasets with many variables, but has relatively few patient samples.

Tests using patient data may consistently outperform other AI methods in both sensitivity and consistency. It was applied to 1,000 blood, namely 352 patients with advanced cancer and 648 patients without cancer. For each sample, the researchers evaluated 44 different variable sets. We found that each consisted of a set of biological features such as DNA fragment length and chromosomal abnormalities, and that aneuploidy-based features (abnormal number chromosomes) provide the best cancer detection performance with a sensitivity of 72% (ability) with the ability to identify 98% of patients. This balance is important in real-world medical applications where false positives need to be minimized to avoid unnecessary steps.

“It may provide a powerful way to measure uncertainty and improve reliability, especially in situations where sample sizes are limited but data complexity is high,” says Dr. Joshua Vogelstein, an associate professor of biomedical engineering and lead investigator.

It has also been extended to a companion algorithm called Comight to determine whether combining multiple sets of variables can improve cancer detection.

The researchers applied comight to blood samples from 125 patients with early breast cancer and 125 patients with early pancreatic cancer, analyzed with 500 controls (cancer-free participants). Pancreatic cancer was detected more frequently than breast cancer, but comight analysis suggests that early stage breast cancer may benefit from combining multiple biological signals, highlighting the potential tool for adjustable detection strategies by cancer type.

In the companion study, researchers, Dr. Christopher Deville, assistant professor of oncology, Dr. Samuel Curtis, a postdoctoral researcher at the Ludwig Centre, and their team found that fragmentation signatures of ccfDNA were previously believed to have been believed to have patients with cancer as well as patients. dermatomyositis and vascular diseases such as venous thromboembolism.

Among individuals with abnormal fragmentation signatures, we found an increase in inflammatory biomarkers in all patients, whether they had autoimmune, vascular disease, or cancer. Their results suggest that inflammation, not cancer itself, is responsible for fragmentation signals, complicating efforts to use CCFDNA fragmentation as a cancer-specific biomarker.

To address the challenge of misinterpreting cancer inflammation, the team added characteristic information about inflammation to its training data. The enhanced version was reduced, but did not completely rule out false positive results for non-cancerous diseases.

“Our main goal was to further explore the biological mechanisms responsible for fragmentation signatures that were previously thought to be unique to cancer,” Curtis says. “As the field moves to more complex biomarkers, understanding the underlying biological mechanisms leading to outcomes is particularly important for interpretation to avoid false positive results. Our new data suggest that patients with non-cancer diseases can be mistakenly believed to have cancer unless appropriate protective guards are incorporated into the test.”

“The silver lining of this study is that it could lead to another diagnostic test for inflammatory disease.”

Together, the research demonstrates the promise and complexity of developing reliable clinical technologies using AI. In a related editorial, the researchers focused on several key challenges that need to be addressed so that they could be fully integrated into clinical practice.

They identified eight important barriers to making AI a routine clinical care. Simply put, these include the false expectation that AI tools must be perfect before they can be considered useful. The need to present the results as probability rather than a simple yes-no-no answer. Ensure that the AI prediction matches the actual probability. Ensure that the results are reproducible. A training model for diverse populations. Explain how AI makes decisions. Recognize how the accuracy of the test changes when illness is rare. Avoid excessive reliance on computer-generated recommendations.

“It could be applied to any field that measures uncertainty and is confident in the reliability and reproducibility of the findings. This could be in the natural, social, or medical sciences. Research in all fields of science requires confidence that what the algorithm spits out is realistic, reproducible and reliable.”

Researchers say results obtained using AI technology should be considered AI-based data that cannot be complemented and cannot be complemented by clinical judgment. Might and Comight provide powerful new tools in cancer detection and potentially inflammatory and vascular disease detection, but say further clinical trials and validation are needed before extending such testing to clinical use.

“With confidence in results being essential and there are reliable quantitative tools out there, we and other researchers can use it and focus on adding more patient research and statistically meaningful features to previous cancer detection tests.”

Might and its companion algorithm, Comight, are published on Treeple.ai.

This study was a collaboration with Vietnamese researchers led by Lan Ho-Pham and Tuan Nguyen, providing important clinical data, samples and interpretations for the study.

Additionally, Joshua Vogelstein, Double, Curtis and Bert Vogelstein, and Johns Hopkins researchers include Tishan Liu, Sambit Panda, Adam Lee, Heyin XU, and Yuxin Bai, Administrator LI, Lisa Doveen, Maria Popoli, Janine Patak, Maxim Petachrosen, Christopher Mekoli, Ami Shah, ogihara Island, Eliza O'Reilly, Yukuan Wang, Michael Goggins, Tian Li Wang, Tian Si, Amanda Fader, Anne Marie Lennon, Ralph Fulvan, Chetan Betgouda, Kinnet Kinzler and Nikka Padaddas research team also included investigators from the University of Pittsburgh, the University of Texas MD Anderson Cancer Center and NYU Langone in the US, University of Melbourne in Australia, Saigon Centre for Precision Medicine, Fam NGOC Tach University, and Tam ANH Institute in Ho Chi Minh City, Vietnam. University of New South Wales. McGill University Health Centre in Montreal. University of Amsterdam Medical Center. In addition to Catmal and Feles, the editorial was written by Elliot Fishman, Bert Vogelstein and Joshua Vogelstein.

These studies were supported by National Institutes of Health Grants R21NS113016, RA37CA230400 U01CA230691, U01CA230691, 5P50CA062924-22, T32GM119998, and tumor core CA 06973. 1R21A1766764-01; Virginia and DK Ludwig Fund for Cancer Research. Lustgarten Foundation, Commonwealth Fund; Thomas M. Homan Memorial Cancer Research Fund. Sol Goldman Sequence Facility in Johns Hopkins. Conrad R. Hilton Foundation. Benjamin Baker Donation 80049589;Swims around Baltimore. JHTV Innovation Grant, The Burroughs Wellcome Career Award for Bedical Scientists; Thomas M. Homan Memorial Cancer Research Fund. National Health and Medical Research Council Investigator Grant App1194970; National Science Foundation NSF Computing Innovation Fellowship 2127309 and Awards DMS-1921310; Rheumatology Research Foundation Investigator Award. Harrington Discovery Institute Scholar Innovator Award Jerome L. Green Foundation. Cupid Foundation. The Stephen and Rennie Bischiotti Foundation.

Bert Vogelstein, Kenneth Kinzler, and Nickolas Papadopoulos are founders of Thrive Anter Detection, an accurate science company. Kinzler, Papadopoulos, and Christopher Douville are consultants who have successfully detected previous discoveries. B. Vogelstein, Kinzler, Papadopoulos, and Douville preserve the fairness of science with precision. B. Vogelstein, Kinzler, and Papadopoulos are founders of Haystack Oncology and Manat Bio and are uniquely fair. Kinzler and Papadopoulos are consultants at Neophore. Kinzler, B. Vogelstein, and Papadopoulos are consultants and consultants for Cage Pharma. B. Vogelstein is a consultant at Catalio Capital Management and holds capital. Chetan Bettegowda is a consultant for Depuy-Synthes, Bionaut Labs, Haystack Oncology, and co-founder of Orisdx. Bettegowda and Douville are co-founders of Diagnosis. The above companies, like other companies, have licensed previously described technologies related to the work described in this paper at Johns Hopkins University. B. Vogelstein, Kinzler, Papadopoulos, Bettegowda, and Douville are the inventors of several of these techniques. Licenses for these technologies are associated with the inventor and the payment of shares or royalties to Johns Hopkins University. Patent applications for the work described in this paper may be filed by Johns Hopkins University. The terms of all these arrangements are governed by Johns Hopkins University in accordance with its interests conflict policy.