{"id":4439,"date":"2026-02-10T09:40:11","date_gmt":"2026-02-10T09:40:11","guid":{"rendered":"https:\/\/salarydistribution.com\/machine-learning\/2026\/02\/10\/new-relic-transforms-productivity-with-generative-ai-on-aws\/"},"modified":"2026-02-10T09:40:11","modified_gmt":"2026-02-10T09:40:11","slug":"new-relic-transforms-productivity-with-generative-ai-on-aws","status":"publish","type":"post","link":"https:\/\/salarydistribution.com\/machine-learning\/2026\/02\/10\/new-relic-transforms-productivity-with-generative-ai-on-aws\/","title":{"rendered":"New Relic transforms productivity with generative AI on AWS"},"content":{"rendered":"
\n

New Relic Inc.<\/a> is a San Francisco-based technology company that pioneered application performance monitoring (APM) and provides comprehensive observability solutions. Serving leading customers worldwide, including major brands like Ryanair, New Relic helps organizations monitor and optimize their digital systems to deliver better customer experiences.<\/p>\n

New Relic faced a challenge common to many rapidly growing enterprises. Their engineers were spending valuable time searching through fragmented documentation across multiple systems, with time consuming internal system queries, in some cases, taking more than a day. As a leading observability platform supporting thousands of customers worldwide, New Relic knew a more efficient way to access and utilize organizational knowledge was needed.<\/p>\n

This challenge led to the creation of New Relic NOVA (New Relic Omnipresence Virtual Assistant): an innovative artificial intelligence (AI) tool built on Amazon Web Services (AWS). New Relic NOVA has transformed how New Relic employees access and interact with company knowledge and systems.<\/p>\n

Working with the Generative AI Innovation Center<\/a>, New Relic NOVA evolved from a knowledge assistant into a comprehensive productivity engine. New Relic NOVA is built on AWS services including Amazon Bedrock<\/a>, Amazon Kendra<\/a>, Amazon Simple Storage Service (Amazon S3)<\/a>, and Amazon DynamoDB<\/a>. Through Strands Agents<\/a>, New Relic NOVA provides intelligent code reviews, AI governance, and managed Model Context Protocol (MCP) services.<\/p>\n

Amazon Bedrock is a fully managed service that provides access to leading foundation models for building generative AI applications, eliminating the need to manage infrastructure while enabling teams to customize models for their specific use cases. Through a single API, developers can experiment with and evaluate different foundation models, integrate them with enterprise systems, and build secure AI applications at scale.<\/p>\n

The solution has reduced information search time while automating complex operational workflows. Through collaboration with the Generative AI Innovation Center, New Relic NOVA was developed into a solution that now processes over 1,000 daily queries across their organization. New Relic NOVA integrates seamlessly with Confluence, GitHub, Salesforce, Slack, and various internal systems, maintaining 80% accuracy in its responses for both knowledge-based queries and transactional tasks.<\/p>\n

We will show how New Relic NOVA is architected using AWS services to create a scalable, intelligent assistant that goes beyond document retrieval to handle complex tasks like automated team permission requests and rate limit management. We explore the technical architecture, development journey, and key lessons learned in building an enterprise-grade AI solution that delivers measurable productivity gains at scale.<\/p>\n

Solution overview<\/h2>\n

In designing New Relic NOVA, New Relic established several critical objectives beyond the initial goal of improving documentation search. These included maintaining data security during knowledge retrieval and achieving consistent response quality across different data sources. As shown in Figure 1, New Relic NOVA\u2019s AWS architecture enables seamless interaction between users and various AWS services while maintaining security and scalability. The solution required a flexible framework that could evolve with the organization\u2019s needs for both knowledge retrieval and transactional tasks. A key challenge was balancing these requirements while keeping response times under 20 seconds to maintain user engagement.<\/p>\n

\"\"<\/a><\/p>\n

Figure 1 \u2013 Solution architecture of New Relic NOVA framework<\/em><\/p>\n

The development team identified several potential risks early in the project. These included the possibility of exposing sensitive information through AI responses, maintaining accuracy when retrieving from multiple data sources, and ensuring system reliability at enterprise scale. Figure 2 illustrates New Relic NOVA\u2019s detailed agent workflow, demonstrating how queries are processed and routed through various specialized agents to address user intentions. Additionally, the team implemented comprehensive security controls which included personable identifiable information (PII) detection and masking, along with a robust evaluation framework to monitor and maintain response quality.<\/p>\n

\"The<\/a><\/p>\n

Figure 2 \u2013 New Relic NOVA agent workflow architecture <\/em><\/p>\n

The project also revealed opportunities for future optimization. These include expanding an agent hierarchy architecture to support additional automated workflows and developing more sophisticated analytics for tracking user interaction patterns. The team\u2019s experience suggests that organizations undertaking similar projects should focus on establishing clear evaluation metrics early and building flexible architectures that can accommodate evolving business needs.<\/p>\n

Solution<\/h2>\n

New Relic NOVA was developed over an eight-week period, involving a collaborative effort between internal engineering, security, legal, and compliance teams and the AWS Generative AI Innovation Center. This partnership accelerated rapid development and iteration, leveraging AWS expertise in large-scale AI implementations.<\/p>\n

Agent architecture<\/h2>\n

The New Relic NOVA architecture consists of three key layers:<\/p>\n

    \n
  1. Main agent layer \u2013<\/strong> This acts as a controllable orchestration for executing different workflows by identifying the user intent and delegating efforts to the following downstream layers:\n
      \n
    1. Retrieval Augmented Generation (RAG) with customized ingested knowledge from Amazon Bedrock Knowledge Bases<\/a> or Amazon Kendra<\/a>.<\/li>\n
    2. Agents for direct interaction with third-party platforms.<\/li>\n
    3. Customized agents for handling internal New Relic tasks.<\/li>\n
    4. Fallback handling if users\u2019 responses cannot be determined.<\/li>\n<\/ol>\n<\/li>\n
    5. Data source layers (vector DB, enrich, data sources)<\/strong> \u2013 These layers represent resources where internal knowledge (for example, New Relic standards documentation and code repository documentation) are ingested for retrieval or RAG purposes. The benefit of these custom resources is to enhance information and search performance for use information requests.<\/li>\n
    6. Agents layer <\/strong>\u2013 Comprises two distinct agent types:\n
        \n
      1. Strands Agents with MCP:<\/strong> Handle multi-step processes for third-party services, leveraging MCP for standardized service interactions.<\/li>\n
      2. Custom action agents:<\/strong> Execute New Relic-specific tasks such as permission requests and service limit modifications, providing precise control over internal systems.<\/li>\n<\/ol>\n<\/li>\n<\/ol>\n

        A central agent acts as an orchestrator, routing queries to specialized sub-agents in a delegation model where responses flow directly back to the user rather than requiring inter-agent reasoning or adjustments. Meanwhile, Strands Agents are used to efficiently manage third-party service integrations using MCP. This approach gives New Relic NOVA the best of both worlds: the orchestration model maintains flexibility for internal processes while standardizing external services through MCP, creating a scalable foundation for New Relic regarding future automation needs.<\/p>\n

        Data integration strategy<\/h2>\n

        The power lies in the ability of New Relic NOVA to seamlessly integrate multiple data sources, providing a unified interface for knowledge retrieval. This approach includes:<\/p>\n