A conversation with PhRMA’s Dr. Gillian Buckley and Life Science Connect’s Jon O’Connell
The spread of artificial intelligence into everyday life has been shaped by Silicon Valley’s “move fast and break things” philosophy. This idea is anathema in a GMP environment where each component must be verified and trusted.
The pressing question for manufacturers, regulators, and technology developers is: can and should they trust AI models when product quality, process control, and patient safety are at stake? Pharmaceutical companies that want to connect artificial intelligence to their bioprocess production lines need to prove that the artificial intelligence is fit for its intended use, is under control, and will continue to perform as expected over time. These important issues are neatly encompassed by the two concepts of trustworthiness and trustworthiness.
Dr. Gillian Buckley, Senior Director of PhRMA, will speak on both establishments in AI models at the inaugural 2026 ISPE AI in Life Sciences Summit — Powered By GAMP at the end of June.
Ahead of the summit, Buckley gave us a preview and explained the difference between reliability and dependability, the importance of usage context, and how to recognize and manage model drift, among other important considerations.
Can you explain the role of reliability and dependability in the context of GMP AI manufacturing tools?
Buckley: Trustworthiness and trustworthiness are related but different things. Trustworthiness concerns the quality of the data used to train a model, whether it is fit for purpose, and whether it supports transparency about how the model works and evolves. Trustworthiness becomes a more important feature when models operate with greater autonomy, making or influencing decisions without direct human approval. Meaningful human oversight remains the norm in today’s pharmaceutical manufacturing. It’s also important to note that machine learning models generally start with a strong baseline of confidence because their output is reproducible and generalizable. Together, these attributes influence the overall risk assessment of the model.
Can you explain what the monitoring program for AI tools in a GMP environment looks like? How does a manufacturer know when a trusted model starts to deviate during deployment?
Buckley: Machine learning models generally become more powerful over time as experience with the tool increases confidence in its use. This is related to the answer above. If companies have sufficient insight into how the model works, and the results are reproducible and explainable, understanding of the model should improve over time.
It is also important to note that manufacturers typically have standard operating procedures (SOPs) that describe procedures for monitoring verification, validation, applicability, and model qualification. These SOPs are typically communicated in a pharmaceutical quality system (PQS). The context of use and explainability of the model informs the need for a level of human oversight. Manufacturers also typically implement risk assessment procedures that evaluate the model’s usage plan, training data set and its robustness, model drift limits, and model retraining procedures. Together, these checks provide a signal to humans when identified model drift may need to be addressed.
Could you please explain an example to help prove reliability?
Buckley: Notable examples include models that combine traditional scientific equations with machine learning to monitor what is happening inside a bioreactor. The model can spot early signs that a batch may be going in the wrong direction and suggest corrective actions, such as adjustments to feed or physical conditions, to get it back on track.
This is a reliable story for two reasons. First, you can evaluate how well your model performs against the documented data that supports your predictions. Second, having many batches increases its track record and builds trust. Also note that even small changes in the way a model is used (usage) can affect its reliability. Reliability is evaluated dynamically in relation to the specific job the model is performing.
Do examples like this make us fully trust our models, or do we always deploy them with skepticism?
Buckley: We are working towards a risk-based framework where the level of oversight is commensurate with the risk of the decision and the explainability of the model indicates how much confidence can be placed in its output. The FDA has already developed a framework for the use of AI in drug manufacturing that takes this approach, with the goal of building a common understanding between regulators and industry on how sponsors can use AI to support regulatory decision-making. As the science matures and both parties gain experience with these tools, we expect trust to grow and AI to be more seamlessly integrated into manufacturing. We are not there yet, but the trajectory is positive. And conversations like this are an important part of building that shared foundation.
While AI models can improve over time with more data, change management frameworks are built around static, deterministic software. How are manufacturers navigating this tension?
Buckley: It’s not as new a problem as you might think. AI and machine learning models share similarities with other computational tools that manufacturers have managed for years, and the same best practices apply, including version control, structured development and validation, and rigorous data governance that includes thorough testing of training data and documentation for future reference.
Models are built and tested in a secure and certified environment with the necessary computational infrastructure to support the model. This foundation is critical to using AI in GMP settings. Such controls allow for a risk-based approach to changing controls. Not all updates require the same level of documentation and review. This risk-based thinking allows manufacturers to keep pace with model evolution while maintaining the control required in a GMP environment.
What characteristics of real-time process data have you seen that manufacturers are using to establish trust in their AI tools?
Buckley: Real-time data is one of the most powerful tools manufacturers have to build and maintain trust in their AI models. You can detect faults, flag deviations early, and confirm that your model is behaving as expected. Real-time data is generated during ongoing model maintenance, with formal procedures (SOPs) governing how such maintenance is performed.
In the digital twin bioreactor example we present, the model can detect titer deviations early in the manufacturing process and suggest steps to correct potential deviations identified. When it detects that a batch is lagging, it suggests specific interventions, such as adjusting nutrient concentrations. Manufacturers can also compare the predicted trajectory of no action to what actually happens if they follow the model’s recommendations. This process is repeated over many batches, and each iteration increases the manufacturer’s confidence in the real-time data.
Finally, what do PhRMA members most frequently report as the biggest barrier to trusting AI in GMP manufacturing?
Buckley: The biggest barrier cited by members is the challenge of using innovative tools in a highly regulated environment. Regulators are understandably wary of issuing guidance on AI before seeing sufficient real-world examples of how it is being used, and the industry is reluctant to move forward without clear expectations from regulators. Both sides are waiting for each other.
The way to break this cycle is precisely through the open dialogue that forums like the ISPE AI in Life Sciences Summit enable. Open discussion fosters a common view of the situation, which builds common understanding. We strongly encourage continued engagement with the FDA and other regulatory agencies, as advancing the use of AI in manufacturing will ultimately benefit public health.
About experts:
Gillian Buckley leads the global quality and manufacturing portfolio at PhRMA. Prior to joining PhRMA, he served as Director of Research at the National Academies of Sciences, Engineering, and Medicine, where he led research on a variety of topics including health care financing, antimicrobial resistance, infectious diseases, and global health. She holds a master’s degree in public health and a Ph.D. Both have doctorates in human nutrition from Johns Hopkins University.
