San Francisco, California, June 12, 2024 – Databricks, a data and AI company, today announced several innovations in Mosaic AI to help customers build production-quality Generative AI applications. Databricks is investing in Mosaic AI in three key areas: support for building complex AI systems, capabilities to improve model quality, and new AI governance tools. The resulting innovations enable customers to confidently build and measure production-quality applications, delivering on the promise of Generative AI for their business.
Organizations struggle to move their Generative AI projects from pilot to full-scale production due to privacy, quality, and cost concerns. Although the underlying models have all made significant improvements, they still struggle to produce high-quality results. Even the best-performing models can return inaccurate and insecure responses or expose sensitive data. To address these challenges, organizations are deploying composite AI systems rather than just deploying one very large model. This approach uses multiple components, including different models, retrievers, vector databases, and tools for evaluation, monitoring, security, and governance. As a result, composite AI systems deliver much higher production quality and enable organizations to efficiently deliver more accurate, secure, and well-governed AI applications.
“We believe that complex AI systems are the best way to maximize the quality, reliability, and measurement of AI applications going forward, and could be one of the most important trends in AI in 2024,” said Matei Zaharia, co-founder and CTO at Databricks. “Databricks is uniquely positioned to capitalize on these trends by investing in improving quality and augmenting models with real-time data, agents, and tools to provide new, little-known capabilities to models.”
To enable customers to build production-quality Generative AI applications, Databricks is releasing the Mosaic AI Agent Framework, Mosaic AI Agent Evaluation, Mosaic AI Tools Catalog, Mosaic AI Model Training, and Mosaic AI Gateway.
The Mosaic AI Agent Framework and Mosaic AI Tools Catalog help organizations build complex AI systems.
Databricks is introducing several new features to help customers deploy enterprise-ready composite AI systems. RAG is a type of composite AI system that uses multiple components, such as a vector database, and monitoring, evaluation, security, and governance tools to improve the accuracy of LLM. Last month, Databricks announced the general availability of Mosaic AI Vector Search as a serverless vector database seamlessly integrated into its Data Intelligence Platform. Today, Databricks announced the Mosaic AI Agent Framework, which enables developers to quickly, securely, and easily build high-quality RAG applications using foundational models and enterprise data. Developers can evaluate the quality of their RAG applications, iterate quickly, and easily redeploy their applications using Mosaic AI Agent Evaluation. Mosaic AI Agent Evaluation is an AI-assisted evaluation tool that automatically determines if the output is of high quality and provides an intuitive UI to get feedback from human stakeholders. Collectively, these capabilities help organizations deploy production-quality generative AI solutions.
Complex AI systems often leverage tools as features to give the system new capabilities to interact with the world, such as intelligently generating and running code, searching the web, calling APIs, etc. The Mosaic AI Tools Catalog enables organizations to manage, share, and register tools using the Databricks Unity Catalog, enabling tool-enabled models to use tools in a secure and governed way and making these tools discoverable across the organization.
Mosaic AI model training enables fine-tuning of underlying models, improving model quality and reducing costs.
Mosaic AI model training uses an organization's private data to fine-tune open source foundational models, delivering new knowledge specific to a domain or task. These fine-tuned models produce higher quality results for specific use cases because they are fully owned and managed by the customer and trained for specialized tasks using the organization's private data. Small models fine-tuned through model training are not only more accurate in a specific domain, but also faster and at a lower cost than large proprietary models because they have fewer parameters and require less computing power.
The Mosaic AI Gateway provides governance for all GenAI apps and models.
Mosaic AI Gateway provides a unified interface to query, manage, and deploy open source or proprietary models, allowing customers to easily switch between large language models (LLMs) powering their applications without complex changes to application code. It supports usage tracking and guardrails, allowing organizations to track who is invoking their models, set rate limits to control spending from enterprise users, and filter safety and personally identifiable information (PII) regardless of which models are being used. Finally, it provides built-in governance and monitoring to ensure continuous quality.
“Corning is a materials science company. Our glass and ceramic technologies are used in many industrial and scientific applications, so understanding and acting on our data is essential. We built an AI research assistant using Databricks Mosaic AI Agent Framework to index hundreds of thousands of documents, including U.S. Patent Office data,” said Denis Kamotsky, principal software engineer at Corning. “It was critical for us that the LLM-powered assistant could respond to questions with high accuracy, helping researchers find and advance the tasks they're working on. To implement this, we built a generative AI solution using Databricks Mosaic AI Agent Framework, augmented with U.S. Patent Office data. Leveraging the Databricks Data Intelligence Platform, we've seen significant improvements in search speed, response quality, and accuracy.”
“FordDirect is at the forefront of digital transformation in the automotive industry. We are the data hub for Ford and Lincoln dealerships, and we needed to create an integrated chatbot to help dealers evaluate performance, inventory, trends, and customer engagement metrics. Databricks Mosaic AI Agent Framework enabled us to integrate our proprietary data and documentation into a generative AI solution using RAG,” said Tom Thomas, VP of Analytics at FordDirect. “By integrating Mosaic AI with Databricks Delta Tables and Unity Catalog, we can now seamlessly run vector indexing in real time as source data is updated, without touching the deployed model.”
“As a leading global manufacturer, Lippert leverages data and AI to build highly engineered products, customized solutions, and best-in-class experiences,” said Kenan Colson, VP of Data & AI at Lippert. “The Mosaic AI Agent Framework has been a game changer for us because it allows us to evaluate the results of our GenAI applications and demonstrate the accuracy of the output while maintaining full control over our data sources. The Databricks Data Intelligence Platform allows us to deploy with confidence into production.”
availability
Mosaic AI Agent Framework, Mosaic AI Agent Evaluation, Mosaic AI Model Training, and Mosaic AI Gateway are currently in public preview. Mosaic AI Tools Catalog is in private preview. To learn more, visit https://www.databricks.com/product/machine-learning.
Source: Databricks
