NEW YORK, June 27, 2024 — Datadog Inc.a monitoring and security platform for cloud applications, announced the general availability of LLM Observability, which enables AI application developers and machine learning (ML) engineers to efficiently monitor, improve, and secure their Large Language Model (LLM) applications. LLM Observability enables enterprises to rapidly deploy and confidently scale generative AI applications in production.
While organizations across industries are racing to release generative AI capabilities in a cost-effective manner, the complexity of LLM chains, their non-deterministic nature, and the security risks they pose can pose several challenges when it comes to implementing and deploying them in production.
Datadog LLM Observability helps customers overcome these challenges and confidently deploy and monitor generative AI applications. The new product provides visibility into each step of the LLM chain, making it easy to identify the root cause of errors or unexpected responses like hallucinations. Users can also monitor operational metrics like latency and token usage to optimize performance and costs, assess the quality of their AI applications like topic relevance and toxicity, and gain insights to mitigate security and privacy risks with out-of-the-box quality and safety ratings.
Unlike traditional tools and point solutions, Datadog's LLM Observability offers fast and responsive clustering, seamless integration with Datadog Application Performance Monitoring (APM), and out-of-the-box assessment and sensitive data scanning capabilities to enhance the performance, accuracy, and security of generative AI applications while maintaining the privacy and security of your data.
“WHOOP Coach is powered by the latest and greatest in LLM AI. Datadog's LLM Observability allows our engineering team to evaluate the performance of model changes, monitor performance in production, and improve the quality of Coach interactions. LLM Observability enables WHOOP to deliver and maintain coaching to all members 24/7,” said Bobby Johansen, Senior Director of Software at WHOOP.
“The Datadog LLM Observability solution helps our team understand, debug, and evaluate the usage and performance of our GenAI applications, allowing us to address real problems like providing a positive experience for end users while monitoring response quality to prevent negative interactions and poor performance,” said Kyle Triplett, VP of Product at AppFolio.
“There's an urgent need to adopt new LLM-based technologies, but organizations of all sizes and industries can find it challenging to do so in a way that's cost-effective and doesn't negatively impact end-user experience,” said Yrieix Garnier, vice president of product at Datadog. “Datadog LLM Observability gives teams the deep visibility they need to manage and understand performance, detect drift or bias, and resolve issues before they significantly impact the business or end-user experience.”
LLM Observability helps organizations:
- Evaluate inference quality: Gain visibility into the quality and effectiveness of your LLM application conversations (e.g. failure to respond) to monitor for hallucinations, drift, and the overall end-user experience of your app.
- Identify the root cause: Gain full visibility into the end-to-end trace of each user request and quickly pinpoint the root cause of errors or failures in the LLM chain.
- Cost and performance improvements: Efficiently monitor key operational metrics for your applications across all major platforms, including OpenAI, Anthropic, Azure OpenAI, Amazon Bedrock, and Vertex AI, in a unified dashboard and uncover opportunities for performance and cost optimization.
- Protection from security threats: Use the built-in security and privacy scanner with Datadog Sensitive Data Scanner to instantly protect your applications from hacks and leak sensitive data like PII, emails, IP addresses, and more.
Datadog LLM Observability is now generally available. To learn more, here.
About Datadog
Datadog (NASDAQ: DDOG) is an observability and security platform for cloud applications. Our SaaS platform integrates and automates infrastructure monitoring, application performance monitoring, log management, user experience monitoring, cloud security, and more to provide integrated, real-time observability and security across our customers' technology stack. Organizations of all sizes and across industries use Datadog to enable digital transformation and cloud migration, drive collaboration across development, operations, security, and business teams, accelerate time to market for applications, speed time to problem resolution, secure applications and infrastructure, understand user behavior, and track key business metrics.
Source: Datadog
