Prototyping an AI agent is easy. Shipping something that business users trust and security teams won’t block is where most enterprise projects slow down.
This blog walks you through a fast and managed path to production using the Databricks platform.
- Build production-grade, domain-specific agents with built-in evaluation and continuous improvement
- Deploy a customizable chat UI using Databricks Apps with built-in SSO and managed data access.
- Distribute agents to business users through a streamlined and intuitive experience to leverage AI and data insights.
One shared example will be used throughout. This is a sample company named Agent Bricks Knowledge Assistant. redwood commerce Answer company policy questions based on internal PDFs and cite source documents.
Why bringing agents into production remains difficult
Teams developing enterprise AI agents often encounter a set of common problems.
- Difficult to evaluate: Many enterprise AI tasks are difficult to evaluate for both humans and automated LLM adjudicators. Academic benchmarks don’t translate to real-world use cases. Building nuanced ratings often requires costly manual labeling. As a result, promising projects stall in endless cycles of adjustment, and stakeholders lose confidence due to unclear progress.
- Too many knobs: Agents are complex AI systems made up of many components, each with its own knobs. From tuning prompts to index chunking strategies to model selection and parameter tweaks, each adjustment has an unknown impact on the entire system. What should be rapid iterative improvement becomes costly, tedious manual trial and error, slowing time to production.
- Cost and quality: Even after a team solves the above problems and builds a high-quality agent, they are often surprised to find that the agent is too expensive to scale into production. Teams end up getting stuck in lengthy cost optimization processes or forced to make trade-offs between cost and quality.
Additionally, you need an intuitive UI for business users and secure access that takes into account your governance model.
The goal is to reduce this friction and enable you to move from proof of concept to business-ready in days or even hours instead of months.
Fast, managed paths: Agent Bricks, Databricks Apps, Databricks One
To deploy AI agents into production, Databricks provides three seamlessly integrated components.
- agent brick Streamline building, evaluating, and optimizing production-grade AI agents against your enterprise data. Just define your tasks and connect your data, and Agent Bricks handles the heavy lifting, including built-in assessments and integrated Unity Catalog governance.
- data brick app You can securely deploy these agents and customizable chat interfaces within Databricks. Get serverless computing, built-in SSO, and granular permissions without having to manage cloud infrastructure.
- databrix one It provides a simplified and curated “front door” for business users. Instead of searching internal wiki pages or managing dashboard bookmarks, you get an intuitive hub for interacting with apps, dashboards, and other data and AI assets.
Let’s take a look at how these three components work together in practice.
Example: Building a corporate policy assistant
The fictitious company Redwood Commerce has company policy documents (travel, expenses, sick leave, and IT security) stored as approved PDFs.
Employees keep asking questions like: “Can I cover hotel dry cleaning as an expense?”
Business users want a simple chat experience that:
- Response based on approved company policy document
- Provides citations for trust and verification
- Respect authority and governance
- Can be widely and securely shared with employees across the organization
Step 1: Create a knowledge assistant with Agent Bricks
Agent Bricks supports multiple use cases, including Knowledge Assistant, which turns documents into high-quality chatbots that answer questions and cite sources.
Connect policy documents
Knowledge assistants can use:
For Redwood Commerce, use the simplest path. That is, save the corporate policy PDF to the Unity catalog volume.
Build the agent
In the Databricks workspace UI:
- Go to agent
- [ナレッジ アシスタント]in,[ビルド]select
- Give it a name (e.g. Redwood Policy Assistant) and add a description
- Select Unity catalog file location as knowledge source
- Create an agent
Knowledge Assistant creates an agent endpoint that you can use downstream of your application.
Step 2: Quickly verify quality (and work with SMEs to improve quality)
A common failure mode is to ship an agent that sounds right but is unreliable. Agent Bricks Knowledge Assistant is explicitly designed to return high-quality answers with quotes, which is key to gaining stakeholder trust.
You can test your agent directly in the Knowledge Assistant UI or AI Playground and ask realistic questions.
- “Can I cover hotel dry cleaning as an expense?”
- “How do I report sick leave?”
- “What is the procedure for refunding transportation expenses?”
The agent’s response is based on the document and cites the relevant policy section.
Agent Bricks helps improve agent behavior based on natural language feedback from subject matter experts (SMEs) by providing labeled questions and guidelines.
Guidelines are used to improve agent responses by setting clear expectations for tone, structure, and behavior. These help agents communicate clearly, stay on brand, and ensure they can successfully handle different scenarios. These same guidelines are also used as evaluation criteria to generate a quality score for each response.
Knowledge Assistant agent[例]Add questions to the tab. Use the 3-point kebab menu to invite SMEs to provide labeled questions and guidelines.[権限]to share your knowledge assistant.
Step 3: Deploy the chat UI using Databricks Apps
Once you’re satisfied with the quality of your agents, turn the agent endpoint into something your employees can actually use: a chat experience specifically for Redwood Commerce.
With Databricks Apps, you can deploy completely custom apps or start with pre-built chat templates and customize them to match your branding.
In the Databricks workspace UI:
- Go to “Computing” and select the “Apps” tab
- Select “Create app”
- [エージェント]Select a tab and choose a chat UI template
- Point it to the Knowledge Assistant endpoint.
- Deploy the app
After you deploy your app, you can use the Knowledge Assistant chatbot directly in your app template via the provided app URL.
To create a more branded experience, you can clone the template to your local machine and customize it. With a few simple adjustments, you can create a bespoke chat UI for Redwood Commerce.
Databricks apps have security and governance built in, so you don’t need to develop and maintain custom authentication or authorization code.
The app can only be accessed by authenticated users who sign in using SSO. There is no anonymous or public access. User authentication allows apps to manipulate app users’ identities and enforce fine-grained permissions.
Step 4: Publish to business users through Databricks One
You can distribute your app by simply sending your app’s URL to people. But as more data and AI assets become available to business users, teams need a single, curated place to ensure employees find the right tools.
Databricks One is designed to be that gateway, a simplified UI for business users to access Databricks shared data and AI assets, including Databricks apps.
After you enable Databricks One and configure the appropriate workspace entitlements, you can share Databricks apps with employee groups synced from your identity provider.
An employee opens Databricks One, clicks Policy Assistant, and asks:
“Can I claim late hotel check-out fees as an expense?”
Citations provide answers and governance is consistent end-to-end.
Get started as a delivery agent for business users
Agent Bricks Knowledge Assistant provides a fast, automated path from corporate documents to domain-specific agents, while keeping quality measurable and improving over time through built-in evaluation and optimization.
With Databricks Apps and Databricks One, you can package that agent into a business-ready chat experience and distribute it through selected entry points, with security and Unity Catalog governance enforced end-to-end.
To dig deeper, start with:
