CQC issues guidance on the use of AI in GP services and shares what you'll find when assessing safety and compliance in areas such as procurement, governance, human monitoring, learning from errors, data protection, and staff training.
The evaluator will ensure that the AI tools are sourced in line with relevant evidence and regulatory standards such as DCB0160 and DTAC, and review clinical governance arrangements to check for proper and safe use. “If AI tools are provided by the NHS procurement list, it makes sense to assume that the appropriate developer standards are met,” CQC said.
There should be responsible CSOs and digital leads in AI technology and “related clinical governance.” Guidance for completing relevant training in this regard continues. Evidence for human monitoring, continuous monitoring, and AI output and process assessments are also considered using organizational audits, critical incident logs, or other quality improvement activities. “We need to demonstrate that AI is being used as a support tool. It is not a replacement for human surveillance,” he adds.
Practices require hazard logs and risk assessments for AI tools, as well as systems to report and investigate when things don't work. CQC checks “established systems for reporting and investigating occurrences” such as reporting to software developers and MHRA's yellow card systems to look for evidence of learning from errors. “The lessons learned must be shared internally,” continues. “For example, we learn staff meetings and CSOs externally sharing them with integrated care systems digital leads, or Patient Safety Events (LFPSE) services when patient safety is at risk.”
Regarding consent, CQC should make patients aware that AI technology is being used and that patients are dissenting, and that “explicit consent is not required, as is Tech, Tech, Tech, AI, Tech, as it is appropriate to rely on implicit consent under the obligation to be a common law obligation to be confidential.” It points to ICO guidance to ensure legality in AI. You should also seek assurances about the risk of bias. IT needs to take steps to reduce digital exclusion and staff need to be trained to be competent in using AI tools.
Finally, guidance sharing should be put into practice how third-party vendors have ensured compliance with data protection standards through the Data Security and Protection Toolkit (DSPT), appropriate cybersecurity arrangements, Data Protection Impact Assessment (DPIA), and GDPR.
Patient care AI
Somerset NHS Foundation Trust shared a series of communications explaining to patients how they use technologies such as AI, Ambient Voice, Virtual Nursing, and Generative AI. Andy Maine, chief scientist in data, operational research and AI, aims to educate patients about the use of these technologies to enhance care and to “be transparent about where we use them.” Andy shared Trust's AI policy late last year. It speaks of its commitment to being transparent, along with safety, equality, inclusion and thorough testing.
Earlier this year, HTN now focused on the practicality of AI technology and explored topics such as implementation, adoption, data role, policy, regulation, assessment, and best practices. With the help of expert panelists, we have taken a closer look at examples of AI in health and care. Panelists included Neill Crump, director of digital strategy for the Dudley Group NHS Foundation Trust. Lee Rickles, CIO, Director and Deputy Shiro of Humber Teaching NHS Foundation Trust. Beatrix Fletcher, Senior Program Manager (AI), Guy's and St Thomas'NHS Foundation Trust.
Bedfordshire, Luton and Milton Keynes ICB have received additional funding from the Adult Social Care Technology Fund for the Digitalised Social Care Program; PainChek®, an AI-driven pain assessment app. Once installed on a smartphone or tablet, PainChek® can use AI to recognize facial expressions to “identify and quantify pain levels.” It is currently used to support patients with communication difficulties, dementia, and learning difficulties, gather information, create data profiles, and support medication prescribing.
NHS Greater Glasgow and Clyde, NHS Lothian and AI Evaluation Company Aival have begun testing the technical performance of AI tools as part of a £1 million project, considering that AI is integrated with existing clinical systems and workflows. Funded by Innovate UK, the project aims to assess the safety and effectiveness of AI technologies, to support the assessment of these tools before procurement, and to create a validation framework to support development. “Less invasive and more cost-effective options.”.
A new approach was created using AI simulations and demographic representative patient and public engagement and engagement panels. Developed by Andrew Steele, a volunteer at NHS Golden Jubilee, the approach incorporates AI leading language models and UK census data, generating what Steele calls “a demographically representative virtual panel with a rich presence of specific experiences and health.”
