AI tools lack suicide prediction, research finds

The accuracy of machine learning algorithms to predict suicidal behavior is too high to predict suicidal behavior and does not help prioritize high-risk individuals for screening and interventions, according to a new study published on September 11th. PLOS Medicine Matthew Spittal and colleagues at the University of Melbourne, Australia.

Over the past 50 years, numerous risk rating measures have been developed to identify patients at high risk of suicide or self-harm. Although these scales generally have poor prediction accuracy, the availability of modern machine learning methods combined with electronic health record data has refocused on the development of new algorithms for predicting suicide and self-harm.

In the new study, researchers conducted a systemic review and meta-analysis of 53 previous studies using machine learning algorithms to predict the outcomes of suicide, self-harm, and suicide/self-harm. Overall, the study included medical records of over 35 million people and nearly 250,000 suicides or self-harm treated in hospitals.

The team found that a high percentage of people identified as modest sensitivity and high specificity, or low risk, did not self-harm or die from suicide. Algorithms are excellent at identifying people who do not want to die from self-harm or suicide, but they are generally poor at identifying their will.

Specifically, researchers found that these algorithms misclassify more than half of those who have subsequently presented themselves to health services for self-harm or who have died from suicide as low risk. Of those classified as high risk, only 6% of people died from suicide afterwards, with fewer than 20% resubmitted to medical services for self-harm.

“We found that the predictive properties of these machine learning algorithms are poorer and less than traditional risk assessment scales,” the authors say. “The overall quality of research in this field is poor, and most studies are not at high risk of bias or unknown. There is insufficient evidence to ensure a change in recommendations in current clinical practice guidelines.”

The author states, “There is a rapid growing interest in the ability of artificial intelligence and machine learning to accurately identify high-risk patients of suicide and self-harm. Our research shows that no algorithms have been developed that have been resubmitted to health services for suicide or self-protection treatment and are not predicted to have a substantial false positive rate.

The authors stated, “Many clinical practice guidelines around the world strongly block the use of risk assessments for suicide and self-harm as a basis for assigning effective aftercare interventions. Our study shows that traditional risk assessment tools can help predict future suicidal behaviors to change the evidence that these guidelines change these guidelines.