Audio is important for detecting suicidal thoughts and is key to understanding the mental and emotional state of people experiencing suicidal thoughts. Suicide hotline counselors are trained to quickly analyze changes in voice to better assist callers through a crisis.
But just as no system is perfect, there is also room for error when interpreting a caller's voice. To help hotline counselors better assess callers, Concordia doctoral student Alaan Fissi developed a model of speech emotion recognition (SER) using artificial intelligence tools. did. This model analyzes and encodes the waveform modulation of the caller's voice. He says the model could lead to improved responder performance in real-world suicide surveillance.
“Traditionally, SER has been performed manually by trained psychologists who annotate audio signals, which requires a high degree of time and expertise,” he says. “Our deep learning model automatically extracts audio features relevant to emotion recognition.”
Nfissi is a member of the Center for Research and Interventions on Suicide, Ethical Issues, and End-of-Life Practices (CRISE). His paper was first presented at his IEEE 18 February 2024.th He won the Best Student Paper Award at the International Conference on Semantic Computing in California.
Instantly read emotions
To build the model, Nfissi used a database of actual calls made to a suicide hotline and integrated it with a database of recordings from different people expressing specific emotions. Both sets of recordings were segmented and annotated by a trained researcher, or an actor voicing the recordings, following a protocol tailored for this task.
Each segment was annotated to reflect a specific state of mind, such as angry, neutral, sad, or fearful/concerned/worried. Actor recordings enhanced the range of emotions in the original dataset, underrepresenting states of anger, fear, concern, and anxiety.
Nfissi's deep learning model then analyzed the data using neural networks and gated recurrent units. These deep learning architectures are used to process data sequences to extract local, time-dependent features.
“This method conveys emotions through a time process, which means we can detect emotions by events that occurred before a certain moment. We know what happened and what happened before. We believe that this will allow us to better detect emotional states at specific points in time. ”
According to Nfissi, this model is an improvement on an existing architecture. Older models required segments to be the same length to process, and usually he was in the range of 5-6 seconds. His model uses variable-length management signals and can handle different time segments without the need for hand-crafted functionality.
The results validated Nfissi's model. Accurately recognized her four emotions in the integrated dataset. Correctly identified fear/concern/anxiety 82% of the time. Neutral, 78 percent. Sad, 77%. And they are angry 72 percent of the time.
The model proved particularly good at accurately identifying professionally recorded segments, with success rates of 78 percent for sad and 100 percent for angry.
This research is personal for Nfissi, who had to study suicide hotline interventions in detail while developing the model.
“Many of these people are suffering, and sometimes a simple intervention from a counselor can be of great help. However, not all counselors are equally trained and are able to process the client's emotions. However, it may take some people more time to understand.”
He said he hopes his model can be used to develop real-time dashboards that will help counselors choose appropriate intervention strategies when speaking with emotional clients.
“This gives us hope that the intervention will reliably help and ultimately prevent suicide.”
The paper was co-authored by Professor Nizar Bouguila from Concordia Institute for Information Systems Engineering, along with Wassim Bouachir from Teluk University and CRISE, and Brian Mishara from UQÀM and CRISE.
If you or someone you know is considering suicide, call or text the Suicide Crisis Helpline at 988 or visit the Montreal Suicide Prevention Center.
Please read the cited paper “Unraveling the emotional state of high-risk suicide callers using voice analysis.”
