Andy O'Conner, Era Sciences
Artificial intelligence is rapidly reshaping the pharmaceutical landscape. What once thought was futuristic now lies at our doorstep: clinical trial monitoring tools, digital QC assistants, document generation models, predictive maintenance in manufacturing, and pharmacy decision support. Based on internal investigations from 174 GXP software vendors, almost half have already promoted AI offerings. There are floods, and as a result, it is a fundamental challenge. How can I use an inherently incomplete, probabilistic and opaque model?
For many quality leaders, this feels like a contradiction. Regulators expect a process that is consistent, traceable and well controlled. Machine learning models work by design through stochastic inference based on data patterns. Probabilistic performance is not an abnormality, it is a characteristic of the system. In fact, if a model always produces the same results, it is not classified as machine learning at all. So how do we adjust this inconsistency?
The Temptation of the Blind Trust
In early conversations about GXP AI, it was common to hear simple slogans such as “Just trust mathematics” and “Models are smarter than us.” These are reassuring, but they don't help you validate your model for real performance. Trust without verification was not the basis of quality control. Trust is acquired through evidence that is not blindly recognized.
Instead, we need a framework that can measure what the model contributes, where it adds risks, and how to mitigate these risks in a controlled way. Fortunately, regulators are beginning to clarify such a framework.
The evolution of the FDA
In the 2025 draft guidance Considerations for the use of artificial intelligence to support regulatory decisions for drugs and biological productsthe FDA proposed Risk-based reliability assessment framework It is built on three pillars:
- Usage Context: Defines the issues of interest and intended use. Context is everything: The model used to prioritize case reports for pharmacobiligallance has a very different risk profile than models that suggest dosage decisions in clinical trials.
- Model risk: Look at the risk The impact of the model (How much AI is shaking in the overall process) The result of the decision (What if the model is incorrect?) Together, these determine whether AI introduces important new risks that need to be mitigated.
- Reliability assessment plan: Establish evidence to compare human-only baselines with human+AI processes. The FDA is explicit. Do not evaluate the model alone. Evaluate the method System with AI Perform in relation to existing practices.
This comparison framing is important. The model doesn't have to be perfect. When combined with human monitoring, while still accepting and controlling risk, it is necessary to clearly improve the performance of regulated processes.
ISPE Gump's perspective
New ISPE GAMP Artificial Intelligence Guide Echo this view. It emphasizes that AI-enabled computerized systems must be in line with Core Gamp 5 principles, including product and process understanding, lifecycle approach, scalable verification activities, science-based quality risk management, and supplier involvement.
Additional AI-specific expectations are overlaid on top: Fits AI literacy objective data, data, model governance, and knowledge management. The quality of the training data is important. This is because biased or inadequately curated data directly shapes the model output. Governance provides a standardized approach to implementing, monitoring and updating models, ensuring decisions remain trackable and compliant. Finally, shared vocabulary and sensual knowledge management build AI literacy across quality, IT, and operations, reducing silos and enabling consistent monitoring.
They can control and accept statistical defects as long as they can understand and explain them well.
“According to the GAMP Artificial Intelligence Guide, explanability is the degree to which it can explain the basis of a decision or action, or how it reaches an output or outcome in a way that a person can understand..
Why is “imperfection” important?
Let's use this as an example to ground. Vendors provide models designed to aid in quality deviation classification. The model takes about 80% of the time. If displayed alone, it sounds like a failure in grades. However, the consistency of deviation classification has resulted in close to 65% of human-only processes between sites where turnarounds are significantly behind, the human+AI process has resulted in better classification, faster triage, and measurable reductions in the backlog.
“Incomplete” AI has improved its value. System Results Without taking away human accountability. Importantly, the comparative assessment gave management confidence that this was not a gamble, but an improvement based on explainable, controlled evidence.
High-quality concept of GXP AI
So, how should quality leaders think about this? Several principles stand out:
- Anchor in the context of usage: Not all AIs are equal. Models that are critical of safety require deeper scrutiny than supporting office productivity. If a person in the process needs to understand the limitations of a model, they will consider their AI literacy and share a model card explaining how to use it.
- Focus on process improvements as well as model performance: Compare humans only with human + AI (not just humans and AI) and focus on realizing and improving the quality and performance of the actual process.
- Quantify both profit and risk: Measures the benefits of efficiency, accuracy, or consistency along with the outcome of potential obstacles. Risk mitigation controls for the use of AI models may appear different to existing mitigation commonly seen in non-AA processes. For example, it should include drift monitoring, periodic reassessment, inappropriate use alerts, specific downstream controls, and more.
- Seek for explanation through actual influences: Perfection is not required, but it has measurable and explainable benefits.
- govern systematically: Uses data and model governance framework, scalable validation, and supplier monitoring that matches GAMP principles.
Value Proposal
The quality of the pharma is always about evidence, not a promise. The same applies to AI. Although models may be probabilistic, our approach to adopting them must remain structured, transparent and documented. If human + AI processes can be demonstrated and explained that they are superior to human-only processes, then imperfection is not considered to break the contract. It's progress.
The focus of GXP AI in Pharma is whether the Human+AI process will deliver better results for patient safety, product quality and data integrity while complying with GXP expectations. If so, even an incomplete model can be justified, documented and implemented.
The current baseline is already incomplete and the structured evaluation allows AI to move that baseline forward.
FAQ
1.How is artificial intelligence used in today's pharmaceutical industry?
AI is increasingly integrated into the pharmacy business, including clinical trial monitoring, digital quality management assistants, generative documentation tools, predictive maintenance, and pharmacovigilance decision support. According to internal research, almost half of the 174 GXP software vendors have already promoted AI features, indicating rapid change.
2. Can AI be compliant with GXP regulations despite being probabilistic and incomplete?
Yes, AI can be GXP compliant when used within a structured risk-based framework. Regulators like the FDA emphasize that the model doesn't have to be perfect. Instead, performance should be assessed in the context of use, with risks understood and mitigated. Even if the model itself is not perfect, the human + AI process that improves results while maintaining control, traceability and compliance is acceptable.
3. What is the FDA's guidance on using AI in regulated drug decisions?
The FDA's 2025 Draft Guidance outlines the three-part framework.
- Usage Context: Clearly define how the model is used and what decisions it affects.
- Model risk: Evaluate the impact of AI decisions and the outcomes of incorrect.
- Reliability assessment plan: Comparing the performance of the AI+ human system with existing human-only baselines.
This comparative, evidence-based approach emphasizes process improvement over isolated model assessments.
4. What does ISPE Gump Guide say about AI in computer systems?
ISPE Gump Artificial Intelligence Guide AI systems reinforce that they must be in line with GAMP 5 principles such as scalable verification, science-based risk management, and supplier monitoring. Add AI-specific guidance.
- High quality, appropriate purpose training data
- Robust Data and Model Governance
- We shared knowledge across AI literacy and functions.
These ensure that decisions made using AI are explained, traceable and compliant.
5. Why is explanability important when using AI in a pharma quality system?
Explanability means being able to clearly explain how an AI-driven decision was made. This is essential for GXP compliance and trust. Incomplete models are acceptable when their effects are well understood and beneficial. For example, models that improve deviation classification from 65% to 80% accuracy show faster turnarounds and measurable values rather than impair quality monitoring.
A version of this article was first published ERA Science's Blog. It has been reissued here with permission.
About the author:
Andy O'Connor combines over 16 years of experience in risk management and technology within the life sciences industry, focusing on quality and IT governance for companies moving towards clinical manufacturing. He holds a Science Honors degree from Dublin University and frequently presents at industry events such as PDA Annual and ISPE. A practical software developer, Andy contributes to numerous enterprise application projects and regularly hosts client training and workshops on risk and data integrity. Outside of work, he volunteered for the Mountain Rescue team in Wicklow, Ireland, reflecting his commitment to both professionals and community service.
