VAST data and lakehouse analytics supplier Cloudera has a strategic partnership to help GPUs continue to process data-fed AI workloads moving through the combined AI Factory data pipeline.
Cloudera Containerization Data engineering, streaming, analytics, machine learning, and AI data services are integrated with VAST’s AI Operating System (AI OS), its data services stack (DataSpace, DataBase, DataStore, DataEngine, AgentEngine, and InsightEngine), all layered on top of a DASE (Distributed and Shared Everything) storage architecture. AIOS, based on Nvidia’s AI data platform reference design, Available on-premises, neocloud, GCP, and Azure environments. PolicyEngine and Tuning Engine were added to AI OS in February, and both are expected to be released by the end of the year.
“Most enterprises already have the data they need for AI. The challenge is to unlock the value of that data and create a continuous pipeline of AI inference, fine-tuning, and data analysis to build the next generation of intelligent applications,” said Jeff Denworth, co-founder of VAST Data.
According to the companies, VAST’s AI OS will transform latent enterprise data into energized, AI-enabled data, on which Cloudera will provide data engineering, analytics, governance, and AI services. The two say they offer the following services:
-
Unified AI factory architecture from raw data ingestion to model deployment
-
Consistent operations across data centers, private clouds, and public clouds
-
Eliminate GPU starvation with ultra-high bandwidth, low-latency data pipelines
-
By forcing the GPU to operate at sustained utilization levels, computing efficiency has been significantly improved.
-
Superior storage performance at scale for structured, unstructured, and multimodal datasets
-
Protect your private AI environment with enterprise governance and compliance
This means customers can implement “production-grade AI systems that continuously transform enterprise data into actionable intelligence for training, inference, analytics, and agent applications” that can be applied to regular private AI, sovereign AI, and more. You can now accelerate your Apache Spark workloads. Nvidia cuDF Accelerate Cloudera Data Engineering workloads and support further optimization of AI workloads by enabling Spark jobs to leverage VAST data services through GPU-accelerated processing.
“Enterprises invest billions of dollars in GPUs, but many struggle to achieve full utilization due to data bottlenecks,” said Abbas Ricky, chief strategy officer, chief business officer, and general manager of applied AI at Cloudera. “Our partnership with VAST eliminates GPU exhaustion and enables our customers to build true AI factories, where data flows seamlessly from ingestion to insight.”
Cloudera and VAST AI Factory’s joint offering is immediately available through both companies’ enterprise sales teams and partner ecosystems. Click here for more information.
