Things are changing rapidly in the world of infrastructure technology. It wasn’t that long ago that running a database on Kubernetes was considered difficult. But that was yesterday’s problem. Cloud-native application builders are adept at running their workloads statefully because Kubernetes is a powerful way to create virtual data centers quickly and efficiently.
Last time I wrote about this, I broadened the scope a bit to consider other parts of the virtual data center application stack, especially streaming workloads and analytics.
As these two move into the Kubernetes mainstream, the use case discussion becomes more interesting. Given that you have access to these basic data tools, what do you do with them?
Thankfully, the industry has already chosen the direction of AI/ML workloads, so there’s no need to dig too deep. Driving this is the need for faster and more agile MLOps to support online prediction, also known as real-time artificial intelligence (AI). Companies like Uber and Netflix were early adopters, but there are many great projects in place to help you get started with Kubernetes more quickly.
Functionality provided by Feast
Building and maintaining machine learning (ML) models is moving from the back office to users in production. A feature store acts as a bridge between your data and your machine learning models, providing a consistent way for your models to access data in both offline and online phases. It manages data processing requirements during model training and provides low-latency real-time access to models during the online phase. This ensures data consistency in both phases and satisfies online and offline requirements. Feast is an example of a feature store running on Kubernetes.
It is open source and allows organizations to store and deliver functionality consistently for offline training and online inference. Feast goes beyond traditional databases by offering specialized features such as point-in-time accuracy.
Serving models using KServe
KServe is an API endpoint for deploying machine learning models to Kubernetes, retrieving models, loading them, and determining if they require CPUs or GPUs. Integrates with KNative eventing for scale-out and provides observability features such as metrics and logging.
Best place? Usage is simple. Just point KServe to your model file and it will create the API and take care of the rest. Explanation features provide insight into why decisions were made for each prediction, provide feature importance, and highlight the factors in the model that led to a particular outcome.
It can be used to detect model drift and bias, which is part of the “important but hard” part of machine learning. These features reduce the effort associated with MLOps and build trust in your applications. KServe was recently spun off from the Google KubeFlow project and highlighted by Bloomberg as part of an effort to build an ML inference platform.
vector similarity search
Vector Similarity Search (VSS) is an enhancement to the traditional way of searching data, a machine learning tool that uses vector mathematics to detect how “close” two things are to each other. This is done by the K Nearest Neighbors (KNN) algorithm, which represents the data as vectors.
The data is then vectorized using the CPU-intensive KNN algorithm and indexed to reduce CPU-intensive searches. End-users can use the query her mechanism provided by the VSS server to provide vectors and find close ones. Open source VSS servers that can be deployed on Kubernetes include Weaviate and Milvus. Both provide everything you need to add similarity searching to your application stack.
form a team
Combining the previous article with this article gives you a recipe for a complete stack deployed on Kubernetes. The result that every organization should strive to achieve is increased productivity and reduced costs. According to a recent study, a dataspace leader discovers both when deploying data infrastructure on her Kubernetes.
AI/ML workloads may be just beginning your exploration, so now might be the perfect time to get started on the right foot. All three of his aforementioned areas (Feature Serving, Model Serving, and Vector Similarity Search) are covered in his book Managing Cloud Native Data with Kubernetes, which I co-authored with Jeff Carpenter. A holistic view of AI/ML in the application stack: Real-time requirements will soon become pervasive in most AI applications. Building fast and reliably with Kubernetes is no longer an AI illusion.
Learn more about real-time AI here.
