Databricks is rolling out a new set of controls for AI applications aimed at preventing security breaches and ensuring compliance. The company announced the release of a beta version Unity AI Gateway Guardrailsa feature designed to provide flexible monitoring of AI models and agent behavior.
Visual TL;DR. AI risks address Databricks Unity AI. Databricks Unity AI introduces AI gateway guardrails. AI Gateway Guardrails provides pre-built and custom controls. Pre-built and custom controls to prevent data leakage. Pre-built and custom controls to prevent harmful outputs. Pre-built and custom controls for branding adjustments. Preventing data leakage enables secure AI apps. Preventing harmful outputs enables safe AI apps. Brand alignment enables secure AI apps. AI Gateway Guardrails enable secure AI apps.
AI risk: Organizations grapple with risks associated with AI deployment
Databricks Unity AI: New controls for AI applications
AI Gateway Guardrails: Beta release for flexible monitoring of AI model behavior
Pre-built and custom controls: Pre-built and custom controls to protect your AI applications
Data Leak Prevention: Protect sensitive corporate data from being exposed to models.
Prevent harmful output: Prevent AI from producing harmful or objectionable content.
Brand alignment: Ensure AI-generated text aligns with brand guidelines
Governed AI: Fundamental governance and security layers for AI
Secure AI Apps: Protect your AI applications from risk.
Visual TL;DR
The move comes as organizations grapple with the risks associated with AI adoption. Databricks highlights the complexity of managing the use of AI, citing its proprietary AI security framework that lists dozens of risks and controls. LLM guardrails are presented as a basic governance and security layer.
These guardrails serve multiple purposes, including preventing sensitive company data from being exposed to models, preventing AI from generating harmful or objectionable content, and ensuring that AI-generated text aligns with brand guidelines. You can also keep conversational AI focused on specific topics.
Databricks is rolling out a new set of controls for AI applications aimed at preventing security breaches and ensuring compliance. The company announced the beta release of Unity AI Gateway Guardrails, a feature designed to provide flexible monitoring of AI models and agent behavior.
Managing generative AI for marketing
Consider Acme Co., a fictional marketing company that uses an AI assistant to design campaigns. CIOs mandate strict policies such as not including customer personally identifiable information (PII) in prompts, screening for jailbreaks and prompt injections, and prohibiting the generation of harmful content.
Additionally, Acme wants to avoid disparaging competitors in its campaign materials. To accomplish this, the AI Platform team configures the Unity AI Gateway endpoint.
Build a governed AI endpoint
The team selects a generic model and sets up inference tables for monitoring. Map business requirements to specific guardrail types.
Detecting and concealing PII: Sanitize input to prevent PII leakage.
Jailbreak and instant injection: Blocks input that attempts to manipulate the AI.
Block unsafe content: Block harmful or dangerous output.
custom blocks: Tailored guardrails to prevent naming and belittling competitors.
Setting up built-in guardrails includes selecting a type, configuring actions such as editing and blocking, and optionally tuning the evaluator model for performance or cost. Log mode allows you to test new guardrails in real traffic conditions without interruption.
Custom guardrails require more detailed prompts that specify business context, competitor names, and provide some example shots. The effectiveness of these custom guardrails can depend on the evaluation model you choose, and Databricks will suggest iterative improvements based on performance and cost.
Guardrail testing and auditing
The Acme team tests the endpoint using various prompts and observes the guardrail behavior. Prompts containing PII are sanitized, and attempts to jailbreak or generate defamatory content are blocked.
Testing revealed that the custom guardrails needed improvement. Reliability was improved by iterating the prompts and switching to a more competent assessment model such as GPT-5.4-mini. Databricks recommends capturing live traffic data to further tune the precision, recall, cost, and delay of your custom guardrails.
Guardrail activity is logged in an inference table, providing detailed insight into request status, token usage, and evaluator responses. These tables allow you to track guardrail decisions to client calls, allowing you to create reports and dashboards for usage analysis and troubleshooting.
This granular visibility can help you validate user sessions if your guardrails prove too sensitive.
Unity AI Gateway’s LLM guardrails are currently in beta and we recommend that users implement them on endpoints that process sensitive data or customer-facing output.
