What is a degradation model and how does AI fit into it?
The algorithm is a predictive model that takes data such as vital signs, information from electronic medical records, and test results in near real time to predict whether a patient's health is about to deteriorate during a hospital stay. Doctors can't monitor all these data points for every patient all the time, so the model runs in the background and checks these values about every 15 minutes. Artificial intelligence is then used to calculate a risk score about the probability that the patient's condition will worsen, and the model sends an alert to the medical team if the patient's condition appears to be worsening.
What are the benefits of implementing such a model in a hospital?
The big question I want to answer is: How can we use AI to build more resilient health systems in high-stakes situations? There are many ways to do this, but one of the core characteristics of a resilient system is strong communication channels. This model leverages AI, but the actions, the interventions that AI triggers are essentially conversations that might not have happened otherwise.
Nurses and doctors talk and hand-off when changing shifts, but busy schedules and other hospital dynamics make it difficult to standardize these communication channels. This algorithm can help standardize and bring clinicians' attention to patients who may need additional care. When the alert reaches the nurse and doctor simultaneously, a conversation begins about what is needed to ensure that the patient does not refuse until her transfer to the ICU is necessary.
Please tell us about how your team implemented and evaluated the model.
Although we did not create this model, we integrated it into our workflow with a few adjustments. Initially, it sent alerts when a patient's condition was already deteriorating, but that wasn't very helpful. We adjusted the model to focus on predicting transfer to ICU and other indicators of health decline.
We wanted the nursing team to feel deeply involved and empowered to initiate conversations with physicians about coordinating patient care. An evaluation of this tool performed on approximately 10,000 patients showed significant improvements in clinical outcomes. There was a 10.4% reduction in worsening events (defined as ICU transfers, rapid response team events, or codes). A subset of 963 patients with risk scores within the “regression discontinuity window”. This basically means they are on the brink of high risk. In these patients, the clinical course may be less clear to the medical team. This model was particularly helpful for that patient group in encouraging doctors and nurses to work together to determine which patients needed special care.
How have nurses and physicians responded to this new model of integration?
This model is far from perfect. Although the response has been positive overall, alert fatigue is a concern as not all alerts are showing a real reduction. When we validated the model on data from patients prior to implementation, we calculated that approximately 20% of patients flagged by the model will experience a worsening of their symptoms within 6 to 18 hours of her arrival. Currently, it's not a completely accurate model, but it's accurate enough to warrant conversation. This shows that algorithms don't have to be perfect to be effective.
That being said, I would like to improve the accuracy. We need to do this to increase trust. That's what we're working on now.
