The UCSF Department of Clinical Informatics and Digital Transformation (DoC-IT) and UCSF Health plan to develop a real-time, continuous, automated artificial intelligence (AI) monitoring platform for clinical care.
Founded with a $5 million gift from Ken and Cathy Hao, the Monitoring Platform for the Impact of AI in Clinical Care (IMPACC) aims to bridge the gap between the rapid evolution of AI technologies used by clinicians and the need for robust, ongoing evaluation of their effectiveness, safety, and fairness.
Julia Adler-Milstein, PhD, director of the UCSF Division of Clinical Informatics and Digital Transformation (DoC-IT), and Sara Murray, MD, MAS, chief health AI officer at UCSF Health, will lead the collaboration.
Before new AI technologies are deployed, evaluations are conducted to assess whether they can be safely integrated into clinical environments, but once deployed, health systems will need ways to quickly identify issues with real-world performance.
IMPACC aims to meet this need by moving from planned, periodic, manual monitoring of a set of focused measures to real-time, continuous, automated, long-term monitoring across a broad set of measures, with specific criteria for human review and escalation to intervention.
“Building IMPACC marks a major step forward in how we analyze the performance of AI in healthcare,” Murray said in a statement. “As we adopt new AI technologies, this novel, scalable platform will provide our healthcare system with direct, actionable insights into ongoing performance to ensure not only the effectiveness of these new tools, but also the safety and patient benefit of the entire system.”
IMPACC will be used to monitor both the performance and impact of AI tools on targeted outcomes over time, such as whether they improve clinical outcomes for patients, as well as to inform healthcare leaders' decisions regarding extending, refining, or disabling tools in the system. Specifically, it will report on whether a tool is achieving its intended results or if it needs improvement, and alert if the tool is potentially dangerous or at risk of exacerbating health disparities, prompting immediate action as needed.
Once developed and tested, IMPACC will be piloted at UCSF Health using an initial set of current AI tools. The effort will be done in coordination with the UCSF Health AI Oversight Committee, which will make recommendations on safety and effectiveness tools and whether they should be deployed more broadly across the health system. The team will also explore building a dashboard that will allow patients to track when AI has been used in their care.
The philanthropic donation “comes at a critical juncture as the healthcare industry broadly embraces AI in clinical practice,” Adler-Milstein said in a statement. “Through IMPACC and this joint effort, we are poised to improve patient care at UCSF and advance the science on how to evaluate AI tools in real-world use.”
