Diving overview:
- Healthcare workers are using artificial intelligence tools that have not been approved by their organizations, posing a potential risk to patient safety and data privacy, according to findings released Thursday by Wolters Kluwer Health.
- More than 40% of healthcare workers and managers said they knew a colleague was using a “shadow AI” product, and nearly 20% said they had used an unapproved AI tool themselves, according to a survey by the information services and software company.
- Dr. Peter Bonis, Wolters Kluwer’s chief medical officer, said while these unapproved tools may be useful to individual workers, their health systems have not vetted the products’ risks and their governance processes have not been considered. “The question is: what are their safety, what is their effectiveness, and what are the risks associated with them?” he said. “And are they sufficiently recognized by the users themselves?”
Dive Insight:
Shadow AI can pose serious security risks across industries when employees use AI products that are not authorized by their companies, experts say. The covert nature of shadow AI prevents organizational leaders and IT teams from knowing how the tools are being used, creating opportunities for cyberattacks and data breaches.
Cybersecurity is already a challenge for healthcare organizations, as cybercriminals frequently target this area, given their valuable data stores and the risks to healthcare delivery.
Bonis said the risks could be even more severe in the medical field. Accuracy is a concern as AI tools can provide misleading or inaccurate information, potentially harming patients. According to a survey of more than 500 respondents in hospitals and health systems, about a quarter of healthcare providers and administrators cited patient safety as their top concern regarding AI in healthcare.
“Even if humans are meant to be in the loop at some point, there are many ways these tools can misfire, and those misfires may not be properly shut off at the point of treatment,” Bonis said.
Still, the promise of this tool to sift through vast amounts of data, speed up administrative tasks, and help doctors document treatments and find medical information makes AI one of the most exciting technologies for the healthcare sector.
Some workers are turning to shadow AI tools to do this, according to research from Wolters Kluwer, a company that provides its own AI-assisted clinical decision support products.
More than 50% of administrators and 45% of care providers said they used unapproved products to provide faster workflows. Furthermore, nearly 40% of administrators and 27% of providers reported using shadow AI due to the tool’s superior functionality or lack of an approved product.
Finally, more than 25% of providers and 10% of administrators cited curiosity and experimentation as reasons for using shadow AI.
Many healthcare workers are unaware of their organizations’ AI policies. Research shows that administrators are more likely to be involved in developing AI policies than care providers. However, 29% of providers said they were aware of their organization’s core AI policies, compared to 17% of administrators.
Bonis said many healthcare providers have likely encountered AI policies related to AI scribes, which typically record clinician-patient conversations and create clinical notes, as these tools have been widely adopted by health systems.
“That’s why they may say they are aware, but they may not be fully aware of all the things that could be considered AI policy,” he said.
