New Relic announced integration with NVIDIA NIM inference microservices, bringing AI observability to the New Relic platform.
The development aims to simplify the complexities and reduce the costs associated with deploying and monitoring Generative AI (GenAI) applications.
According to New Relic, the integration provides comprehensive visibility into the entire AI stack for applications developed with NVIDIA NIM. The addition is expected to improve the performance and security of AI applications, as well as make the setup process easier.
New Relic CEO Ashlan Willy emphasized the importance of observability in today's competitive environment.
“In today's highly competitive market, organizations can't afford to wait years for an AI return on investment (ROI),” he said.
“Observability solves this problem by providing visibility across the entire AI stack, and NVIDIA is pioneering AI observability by extending our platform to include AI apps built with NVIDIA NIM. NVIDIA's AI technology, combined with our expertise in observability and APM, gives enterprises a competitive advantage in the AI race.”
Peter Marelas, Chief Architect APJ at New Relic, also noted the significant growth of generative AI in the Asia Pacific region. “As Asia Pacific organizations prepare for a significant increase in generative AI users over the next five years, it is critical that they have visibility into their AI apps. Our support for NVIDIA NIM gives organizations real-time insight into how their NVIDIA NIM deployments are performing,” he said.
Amanda Saunders, director of AI software at NVIDIA, added: “As enterprises race to adopt generative AI, NVIDIA NIM helps them quickly deploy applications into production. New Relic's integration with NVIDIA NIM empowers IT and development teams to optimize AI applications by quickly observing and acting on operational insights.”
This integration is specifically aimed at helping organizations adopt AI more quickly and achieve faster return on investment. The integration provides users with capabilities such as full-stack visibility, detailed tracing insights, and important GPU utilization metrics. It also enhances data security by allowing sensitive data to be excluded from the monitoring process.
One of the key features is a holistic view of the entire AI stack, including applications, NVIDIA GPU-based infrastructure, and the AI layer. Users can also remediate performance and quality issues such as bias, toxicity, and hallucinations by tracing the lifecycle of AI responses. Additionally, the platform provides model inventory tracking and performance comparison capabilities to optimize model selection based on infrastructure and user requirements.
New Relic's platform supports a wide range of AI models, including Databricks DBRX, Google's Gemma, Meta's Llama 3, Microsoft's Phi-3, Mistral Large and Mixtral 8x22B, and Snowflake's Arctic.
These models will be able to monitor performance and cost metrics in real time, enabling organizations to deploy AI applications with more confidence and reduce time to market.
This integration is the latest in a series of efforts by New Relic to expand its capabilities in the AI space. Recently, New Relic joined NVIDIA's AIOps partner ecosystem to combine observability with AI, leveraging NVIDIA AI accelerated computing. New Relic AI Monitoring is part of the company's all-in-one observability platform and is offered with a usage-based pricing model.
With this integration, New Relic continues its mission to provide engineers with a comprehensive observability platform, providing insights that improve the planning, deployment and operation of AI applications. The addition of NVIDIA NIM support further solidifies New Relic's position in the growing AI market.
