Artificial intelligence algorithms to predict suicidal behavior are too inaccurate and there is no screening of high-risk people. Systematic reviews and meta-analyses reveal.
Survey results PLOS MedicineDash hopes that modern machine learning methods are refined enough to identify those at the lowest risk of suicide and self-harm due to personalized interventions.
Machine Learning misclassified more than half of people who went to the hospital due to self-harm, or more than half of people who committed suicide and died as low-risk.
“Many clinical practice guidelines around the world strongly block the use of suicide and self-harm risk assessments as a basis for assigning effective aftercare interventions,” reported Dr. Matthew Spittal and colleagues at the University of Melbourne.
“Our research shows that machine learning algorithms are less suitable for predicting future suicidal behavior than traditional risk assessment tools these guidelines are based on. We do not see any evidence to guarantee changes to these guidelines.”
Over the past 50 years, numerous risk rating measures have been developed to identify patients at high risk of suicide or self-harm and to classify them as high or low risk.
Although treatment routes are often based on such risk assessments, these scales generally lack accuracy, and clinical practice guidelines strongly advise against the use of them for suicide and self-harm.
Nevertheless, the advent of modern machine learning methods and access to electronic health records and registry data has led to renewed interest in developing prediction algorithms.
To investigate the accuracy of these new methods, Spittal and the team conducted a systematic review and meta-analysis of 53 studies identified through an online database.
Studies were included when the outcomes of suicide or hospital-treated self-harm were investigated and case control, case cohort, or cohort study design was used.
Overall, researchers reported that research in this field was of low quality, with most being at either a high or unknown risk of bias.
The algorithm was well-accurate when assessed using a global scale using the area under the receiver operating characteristic curve in the range of 0.69 to 0.93, but less accurate when assessed for more clinically relevant individual measurements.
The algorithm had a modest sensitivity between 45% and 82% and had a high specificity of 91% and 95%. The positive likelihood ratio ranged from 6.5 to 9.9, with negative likelihood values ranging from 0.2 to 0.6.
This meant that while they were good at identifying people who were not present again due to self-harm, or who would not die from suicide, they were generally poor at identifying people who were present again.
The slight sensitivity observed in the cohort study indicates that over half of those who repeatedly or die self-harm are misclassified as low risk.
Overall, researchers viewed machine learning algorithms as poor predictive properties and superior to traditional risk rating scales.
They advocate for the management of patients treated in hospitals for self-harm, and should include three factors: Need-based evaluation and response. Identification of modifiable risk factors with treatments aimed at reducing these exposures. Implementation of proven effective aftercare interventions.
“Instead of predicting suicide and self-harm, there may be other ways that artificial intelligence can be used to contribute to better outcomes for suicide patients,” the team proposed.
“Future research can explore ways to use machine learning methods to enhance existing co-psychosocial assessments.
“Specifically, can machine learning methods be used to identify modifiable risk factors for suicide and self-harm in individual patients? This can be a more manageable issue, as the prevalence of many risk factors is likely higher than the prevalence of suicide or self-harm.”
