AI helps identify childhood cancer survivors who need additional support

Artificial intelligence (AI) could help doctors determine whether childhood cancer survivors need additional support. The more information the AI ​​includes in its prompts, the better its performance will be. The discovery was announced today. communication medicine This research by scientists at St. Jude Children’s Research Hospital could lead to the integration of AI into future clinical workflows.

Scientists observed how well a large-scale language model, a type of AI, could analyze interviews with young survivors and their caregivers and detect multiple symptoms that cause serious disruption to daily life. After comparing different prompting approaches, the researchers found that more complex prompts, which provide additional information to the model, performed best. This result suggests that future efforts to leverage AI to improve survivor care should consider these advanced facilitation strategies rather than simpler ones.

“About 40% to 60% of clinical situations involve patients telling their doctors about symptoms and related health experiences,” said corresponding author Dr. I-Chan Huang of St. Jude’s Office of Epidemiology and Cancer Control. “We have provided proof of concept that large-scale language models can help analyze underutilized conversational data, detect symptom severity and its impact on function, and assist physicians in decision-making to provide better care to survivors.”

Comparison of strategies to promote survival

Children with cancer receive treatment at a critical time in their development, which can have ripple effects later in life. Cancer- and treatment-related effects can occur long after the initial illness has been cured. However, it is difficult for doctors to identify survivors whose symptoms are severe enough to require additional targeted support. Much of your personally identifiable data is contained in conversation transcripts and responses to open-ended survey questions and is not readily visible. New language-based AI gives doctors the opportunity to analyze, understand, and use that information to help survivors.

Researchers interviewed 30 survivors between the ages of 8 and 17 and their caregivers. Two human experts analyzed the conversation recordings for signs of excessive pain or fatigue, resulting in more than 800 pieces of analyzable information. They categorized symptoms by severity and their physical, cognitive, and social effects. After performing that gold standard analysis, the scientists fed the same transcripts to two large language models, ChatGPT and Llama, using four styles of prompts. Both models demonstrated the ability to analyze data in a manner similar to experts, but their performance varied depending on the prompts used.

Prompting is a technique that instructs AI to perform a task. The researchers compared four common prompting strategies: two simple and two complex. A simple approach is zero-shot and few-shot prompts, where no or minimal information is provided beyond basic instructions. These approaches produced unstable and inaccurate results.

“We found that simple prompts are not effective,” Huang said. “However, our more sophisticated prompting strategy performed significantly better and had better agreement with human reviewers.”

Two complex strategies were chain of thought and generated knowledge prompts. Although chain of thought uses step-by-step logical instructions, the knowledge generated prompts the model to come up with background information before giving instructions to the model. Both complex prompting methods had moderate ability to detect social effects while successfully identifying the physical and cognitive effects of symptoms on survivors.

Although more testing will be required for clinical use, these initial results suggest that thought chains, knowledge generated, or similar facilitation methods should be used in the future. The findings provide one of the first concrete examples of how AI can improve survivor care.

“These AI-driven approaches offer new ways to uncover complex symptom information hidden in rich patient-physician conversations that are currently untapped,” said Huang. “By making it easier to capture and analyze this information, we can help physicians better identify survivors who need additional support in real time and improve care for this growing population.”

Authors and funders

The study’s lead author is Jin-ah Sim, formerly of St. Jude. Other authors of the study are Madeline Horan, formerly of St. Jude University and currently of Wake Forest University School of Medicine; Xiaolei Huang, University of Memphis. Kim Min-soo, Hallym University. Kumar Srivastava, Kirsten Ness, Melissa Hudson, St. Jude. Justin Baker, formerly of St. Jude University and currently at Stanford University School of Medicine.

This research was supported by grants from the National Cancer Institute (U01CA195547, R21CA202210, R01CA238368, R01CA258193), a Cancer Center Support (CORE) grant (CA21765), and the American Lebanese and Syrian Association of Charities (ALSAC), St. Jude’s fundraising and awareness arm.