TL;DR:
- New Relic introduces New Relic AI Monitoring (AIM), an APM solution for AI-powered applications.
- AIM provides unprecedented visibility into the AI application stack for enhanced troubleshooting and optimization.
- Key features include LLM response tracing, model comparison, and comprehensive AI stack visibility.
- AIM ensures responsible AI use by addressing issues like bias, toxicity, hallucinations, and compliance with AI regulations.
- AIM is available in early access and is seamlessly integrated into New Relic’s consumption-based pricing.
Main AI News:
New Relic, the renowned all-in-one observability platform, has introduced an innovative solution tailored for the age of AI-powered applications – New Relic AI Monitoring (AIM). This advanced Application Performance Monitoring (APM) solution empowers engineers with unprecedented visibility and insights into the AI application stack. AIM facilitates streamlined troubleshooting and optimization of AI applications, addressing key factors such as performance, quality, cost efficiency, and responsible AI utilization.
Boasting an extensive array of over 50 integrations and cutting-edge features like LLM response tracing and model comparison, AIM provides the foundation for engineering teams to develop and operate Large Language Model (LLM)-based applications with unwavering confidence.
In a world increasingly reliant on AI-driven technologies, AIM offers comprehensive observability for all AI-powered applications. Engineers now have access to a unified platform for pinpointing and resolving issues while also comparing and optimizing various LLM prompts and responses. This encompasses performance, cost management, security, and quality concerns, including the critical aspects of hallucinations, bias mitigation, toxicity control, and fairness in AI applications.
AIM delivers complete transparency by offering engineers a holistic view of every facet of the AI stack, alongside the associated services and infrastructure. This equips them with essential data to ensure the responsible use of AI and to demonstrate compliance with forthcoming AI regulations.
Manav Khurana, Chief Product Officer at New Relic, emphasized the significance of this development, stating, “With every organization integrating AI into their products and processes, AI workloads are now part of modern organizations’ application architectures. With AI monitoring, we have applied our deep expertise from inventing cloud APM to providing end-to-end visibility into AI-powered applications to help businesses manage performance, costs, and the responsible use of AI.“
Key Features and Use Cases Include:
- Auto Instrumentation: New Relic agents come equipped with all AIM capabilities, ensuring swift and hassle-free setup, including full AI stack visibility, response tracing, model comparison, and more.
- Full AI Stack Visibility: Gain a comprehensive understanding of the application, infrastructure, and AI layer, with insights into AI metrics, such as response quality and tokens, alongside APM golden signals.
- Deep Trace Insights for Every LLM Response: Trace the entire lifecycle of intricate LLM responses created using tools like LangChain, enabling rapid resolution of performance issues and quality problems, such as bias, toxicity, and hallucination.
- Compare Performance and Costs: Monitor and optimize usage, performance, quality, and cost across all AI models within a single view. Leverage insights on frequently asked prompts, chain of thought, prompt templates, and caches to enhance efficiency.
- Enable Responsible Use of AI: Ensure the safe and ethical deployment of AI by verifying that responses are appropriately tagged as AI-generated and free from bias, toxicity, and hallucinations, utilizing response trace insights.
- Instantly Monitor Your AI Ecosystem: AIM offers the most comprehensive solution for monitoring the entire stack of any AI ecosystem, with a wealth of integrations and quickstarts spanning orchestration frameworks, LLM platforms, machine learning libraries, model serving, vector databases, and AI infrastructure.
AIM is now available in early access to New Relic users worldwide. Users can sign up to request early access, which is seamlessly integrated into New Relic’s simplified, consumption-based pricing structure. Experience the future of AI monitoring with New Relic AIM and elevate your AI-powered applications to new heights of performance, quality, and responsibility.
Conclusion:
New Relic’s AIM signifies a pivotal shift in AI monitoring, empowering businesses to harness the full potential of AI-powered applications while ensuring responsible usage. This innovation is poised to reshape the market by offering comprehensive observability and transparency, essential for the evolving landscape of AI-driven technologies.
