AI model uses doctor’s notes to predict survival after cancer diagnosis

Important points:

• The best AI model had an accuracy of over 80%.
• ‘Breast’ and ‘Prostate’ positively predicted 6-month survival. ‘Liver’, ‘glioblastoma’ and ‘lung’ negatively predicted 60-month survival.

Researchers at the University of British Columbia have developed an artificial intelligence model that can accurately predict survival after a cancer diagnosis using doctors’ notes from their first oncology appointment.

Artificial intelligence (AI) models use natural language processing to predict short- and long-term survival with over 80% accuracy, according to research results published in . JAMA network open.

“Better understanding of treatment outcomes is a key component of personalized cancer care.” John Jose Nunez, MD, MSc, A psychiatrist and clinical research fellow at the University of British Columbia Mood Disorders Center and BC Cancer, he told Healio. I see it as another tool to use when

Background

Despite the recent surge in AI-related publicity, Nunez says he has always been interested in using computational technology to help patients.

One of his group’s goals is to harness the power of AI to go beyond its current commercial exploits and achieve the higher goal of helping cancer patients.

“Oncologists know all the inside and outside statistics when it comes to population-level survival, but there is some evidence that predicting survival becomes more difficult when it comes to the patient in front of us.” said Nunez.

Nunez said researchers sought to develop AI models that would allow for more personalized survival predictions using unstructured data, which offers the potential for greater portability.

methodology

Nunez et al. conducted a retrospective prognostic study of 47,625 patients (mean age 64.9 ± 13.7 years, 53.4% ​​female) who started treatment at a BC cancer facility between April 1, 2011, and December 31, 2016. was carried out.

Researchers obtained patient data and initial oncologist consultation documents from BC Cancer and analyzed them using four previously established language models. This includes one traditional non-neural method language model and three of his models using neural networks.

Investigators obtained administrative mortality data from BC Vital Statistics. Patients diagnosed with multiple cancers were excluded from this analysis.

The AI ​​model’s performance in predicting patient survival at 6, 36, or 60 months served as the study’s primary outcome. Researchers evaluated model performance using common metrics such as precision, specificity, sensitivity, balanced precision, and receiver operating characteristic area under the curve (AUC). Secondary outcomes consisted of the words used in the model and their impact on predicted values.

April 6, 2022 served as the data cutoff date for updated mortality data.

Main findings

Median survival for patients with a minimum of 5 years of follow-up was 61.7 ± 40.3 months after diagnosis and 59.9 ± 39.9 months after first oncology visit across the study cohort.

Starting with the first oncologist visit, researchers found that 41,447 patients (87%) survived 6 months, 31,143 (65.4%) survived 36 months, and 27,880 (58.5%) survived 60 months. reported that it did.

In the holdout test set, the best AI model was 0.856 (AUC = 0.928) for predicting survival at 6 months, 0.842 (AUC = 0.918) for predicting survival at 36 months, and 0.837 (AUC = 0.918) for predicting survival at 60 months. = 0.918). Moon Survival Prediction.

Root analysis of dialogue between recordings and physician notes showed that the words “breast” and “prostate” were positive predictors of 6-month survival, whereas “liver” and “glioma” were positive predictors of 6-month survival. “tumor” and “lung” were negative predictors of survival. 60 months survival.

clinical significance

According to Nunez, the key takeaway from a research perspective is that survival predictions can be made accurately without overwhelming amounts of structured data. Accurate predictions can be derived from unstructured data readily available from electronic medical records in oncologist’s offices.

From a clinical perspective, Nunez said the tool is still years away from widespread adoption and needs further validation using documents from other regions.

“AI will have many interesting tools coming out of the pipeline,” he told Healio. “We encourage clinicians to be open to using them when they become available.”

For more information:

John Jose Nunez, MD, MSc, You can contact johnjose.nunez@bccancer.bc.ca.