Red Hat extends OpenShift for the era of generative AI

IBM’s Red Hat business unit is expanding its AI capabilities with new Red Hat OpenShift AI technology.

The OpenShift AI Platform was announced today at the Red Hat Summit. For the past decade, OpenShift has been Red Hat’s flagship application container product based on the open source Kubernetes container orchestration platform. OpenShift AI is (as the name suggests) a version of the platform optimized to enable AI and machine learning (ML) deployments.

The new platform is an evolution of the Red Hat OpenShift data science platform, focused on enabling production deployment of AI models.

“Over the past 10-20 years, we have focused a lot of our time and energy on building application platforms. We’re focused on that,” said the Red Hat CTO. Chris Wright told reporters and analysts: “The challenge for companies to implement his AI/ML is enormous.”

IBM already uses OpenShift AI

Wright pointed out that the reality for many companies is that data science experiments often fail, with less than half making it to production.

Red Hat’s goal with OpenShift AI is to provide a collection of tools that provide the ability to do everything AI needs to train, deliver, and monitor, in a way that helps more models reach production. That’s it. This is an approach and technology already proven by Red Hat through its parent company IBM.

Wright commented that the cost and complexity of training large language models (LLMs) is particularly high. When IBM started building its new watsonx foundation model (announced earlier this month), it turned to Red Hat OpenShift.

“Our platform is the platform IBM uses to build, train and manage the underlying models to demonstrate the kind of scale and production capabilities IBM has built into OpenShift AI,” he said.

Red Hat is building a set of extensions to OpenShift AI. It also includes model performance features. Wright said OpenShift AI will continue to improve the ability of his data scientists to manage the monitoring and performance of models deployed into production. Part of model performance also involves monitoring potential model drift and ensuring that the model is accurate.

Deployment pipelines for AI/ML workloads are also important. To that end, Red Hat OpenShift AI enables organizations to create a repeatable approach to model building and deployment. There are also efforts to integrate custom runtimes for building AI/ML models.

“One of the things we found is that data science teams are spending a disproportionate amount of time just building tools,” Wright said. “Of course, you can create a set of tools, but it may not be exactly the set of tools the company is looking for, so you may need to customize the runtime environment.”

What AI/ML workloads need to reach production is the ability to integrate AI quality metrics. Wright noted that many data science experiments fail because of a lack of alignment with business outcomes.

Then, “it’s hard to measure your own success,” Wright said. “So I think it’s really important to make sure that we can incorporate metrics throughout the pipeline.”