Dr. Laura Duncan of HHS is leading a study using AI to analyze the voices of teens in McMaster Children’s Hospital programs to better understand their mental health needs.
What’s in the voice? It all depends on what you hear, says Dr. Laura Duncan of Hamilton Health Sciences (HHS). He is leading a study that uses artificial intelligence (AI) to analyze the voices of teens in programs at McMaster Children’s Hospital (MCH) to better understand their mental health needs.
While human listeners can sense tone, emotion, and context in a person’s voice, machine learning can “hear” in a completely different way by turning audio into data, searching for patterns, and detecting subtle nuanced changes in pitch, volume, speed, and tone that humans can’t hear.
Where you can’t hear
Duncan, director of research and information systems for MCH’s Child and Youth Mental Health Program (CYMHP), said AI interprets speech patterns very differently than a human listener. The program collects and analyzes data to better understand young people’s mental health needs and assess how well services are working. We also develop tools to assess children and young people, identify trends and gaps in care, and improve services. Mr. Duncan is also a researcher at the Offord Child Research Center, an affiliated research institution between HHS and McMaster University.
Over a six-month period, teenage patients in a child and youth mental health program used their mobile phones and computers to record themselves describing pictures provided in the study or reading chapters of a story, and shared the audio recordings for AI speech analysis.
Research using electronic information collected by programs involves deep learning, an advanced form of AI that automatically finds patterns in very large or complex data such as images, text, and audio.
In a recent project, we recruited 50 CYMHP patients between the ages of 12 and 17 to use AI voice analysis to better understand their mental health needs. For six months, these patients used their mobile phones or computers to record themselves describing a photo or reading a chapter of a story provided by the study, and shared the audio recordings for AI speech analysis. As part of the project, they also completed daily sleep and mood assessments over a six-month period.
Yushan (Bryce) Lei, a master’s student at McMaster University’s Offord Center, demonstrates how a teenage patient uses a cell phone or computer to record pictures as they describe them or read chapters of a story provided in the study.
While human listeners can sense tone, emotion, and context in a person’s voice, machine learning can “hear” in a completely different way by turning audio into data, searching for patterns, and detecting subtle nuanced changes in pitch, volume, speed, and tone that humans can’t hear.
“We are using AI to investigate whether indicators in a patient’s voice can tell us something about the trajectory of symptoms over the course of treatment,” Duncan said, adding that such insights can help medical teams better understand a patient’s needs and plan the best and most appropriate treatment.
next step
“Our data engineers are currently examining the audio recordings to identify key details that will help us transform the data into a workable model,” Duncan said, predicting they will be ready to share their findings as early as this spring.
This audio study is part of an AI deep learning project called SMARTmind. Advances in mental health with AIlaunched about five years ago and partially funded by the Juravinsky Institute. Duncan is co-principal investigator along with child psychologist Dr. Paulo Pires and former HHS child psychiatrist Dr. Roberto Sassi, now at BC Children’s Hospital.
SMARTmind uses advanced computational tools to examine large scale connected health data for clues that can help predict how often people will need hospital services in the future and how serious those visits will be.
The system builds on online measurement tools already in place to assess and monitor children’s mental health, such as the Mental Health Questionnaire for Children and Youth (MHQ-CY). Duncan helped create and test the online intake assessment tool MHQ-CY, which is the hub of the community-wide information system.
The MHQ-CY includes an online set of questions that young patients, caregivers, and family members complete to describe their symptoms, concerns, and strengths. Such background information helps care teams understand what the young person is experiencing, determine what types of supports and services are needed, and ensure that care is consistent across different providers. It also helps organizations better plan and improve mental health services by showing the most common needs in their communities.
“We have an incredible database that allows us to apply machine learning, especially deep learning, to explore new ways to advance research,” Duncan says, explaining how MHQ-CY is contributing to AI learning.
As advances in technology continue to move research forward, Duncan looks forward to improved connections between local and regional organizations supporting child and youth mental health.
“Having an integrated system across the state will provide the foundation for AI to truly become more effective,” Duncan said.
