{"id":951,"date":"2021-09-28T06:43:22","date_gmt":"2021-09-28T06:43:22","guid":{"rendered":"https:\/\/salarydistribution.com\/machine-learning\/2021\/09\/28\/detect-anomalies-using-amazon-lookout-for-metrics-and-review-inference-through-amazon-a2i\/"},"modified":"2021-09-28T06:43:22","modified_gmt":"2021-09-28T06:43:22","slug":"detect-anomalies-using-amazon-lookout-for-metrics-and-review-inference-through-amazon-a2i","status":"publish","type":"post","link":"https:\/\/salarydistribution.com\/machine-learning\/2021\/09\/28\/detect-anomalies-using-amazon-lookout-for-metrics-and-review-inference-through-amazon-a2i\/","title":{"rendered":"Detect anomalies using Amazon Lookout for Metrics and review inference through Amazon A2I"},"content":{"rendered":"
\n

Proactively detecting unusual or unexpected variances in your business metrics and reducing false alarms can help you stay on top of sudden changes and improve your business performance. Accurately identifying the root cause of deviation from normal business metrics and taking immediate steps to remediate an anomaly can not only boost user engagement but also improve customer experience.<\/p>\n

As the volume of data monitored by your business grows, detecting anomalies gets challenging. On March 25, 2021, AWS announced the general availability of Amazon Lookout for Metrics<\/a>, a service that uses machine learning (ML) to automatically detect anomalies that are most important to businesses with greater speed and accuracy, and identifies their root cause.<\/p>\n

ML models often need human oversight to retrain and continuously improve model accuracy. In this post, we show how you can set up Lookout for Metrics to train a model to detect anomalies. We then use a human-in-the-loop workflow to review the predictions using Amazon Augmented AI<\/a> (Amazon A2I), and use the feedback to improve model accuracy.<\/p>\n

Solution overview<\/h2>\n

Lookout for Metrics uses ML to automatically detect and diagnose anomalies (outliers from the norm) in business and operational data, such as a sudden dip in sales revenue or customer acquisition rates. In a couple of clicks, you can connect Lookout for Metrics to popular data stores like Amazon Simple Storage Service<\/a> (Amazon S3), Amazon Redshift<\/a>, and Amazon Relational Database Service<\/a> (Amazon RDS), as well as third-party software as a service (SaaS) applications such as Salesforce, ServiceNow, Zendesk, and Marketo, and start monitoring metrics that are important to your business.<\/p>\n

Lookout for Metrics automatically inspects and prepares the data from these sources to detect anomalies with greater speed and accuracy than traditional methods used for anomaly detection. You can also provide feedback on detected anomalies to tune the results and improve accuracy over time. Lookout for Metrics makes it easy to diagnose detected anomalies by grouping together anomalies that are related to the same event and sending an alert that includes a summary of the potential root cause. It also ranks anomalies in order of severity so you can prioritize your attention to what matters most to your business.<\/p>\n

\"\"<\/a><\/p>\n

Amazon A2I is an ML service that makes it easy to build the workflows required for human review. Amazon A2I brings human review to all developers, removing the undifferentiated heavy lifting associated with building human review systems or managing large numbers of human reviewers, whether running on AWS or not.<\/p>\n

To get started with Lookout for Metrics, we create a detector, create and attach a dataset with metrics that you want to train on and monitor, activate the detector, and view anomalies. Following these steps, we show how you can set up a human review workflow using Amazon A2I. Finally, it updates the detector with human review feedback, which helps retrain the model and further improve accuracy.<\/p>\n

Architecture overview<\/h2>\n

The following diagram illustrates the solution architecture.<\/p>\n

\"\"<\/a><\/p>\n

The solution has the following workflow:<\/p>\n

    \n
  1. Upload data from your source to Amazon S3.<\/li>\n
  2. We run Lookout for Metrics in continuous mode to process data continuously from the Amazon S3 path.<\/li>\n
  3. Inference results are stored in Amazon S3<\/li>\n
  4. When Lookout for Metrics detects anomalies, the inference input and outputs are presented to the private workforce for validation via Amazon A2I.<\/li>\n
  5. A private workforce investigates and validates the detected anomalies and provides feedback.<\/li>\n
  6. We update the results with corresponding feedback from human loop through Amazon A2I.<\/li>\n
  7. Updated feedback improves accuracy of future trainings.<\/li>\n<\/ol>\n

    In the accompanying Jupyter notebook downloadable from GitHub<\/a>, we walk you through the following steps:<\/p>\n

      \n
    1. Generate a synthetic dataset.<\/li>\n
    2. Create a detector and map measures and dimensions to metrics.<\/li>\n
    3. Activate the detector.<\/li>\n
    4. Detect anomalies.<\/li>\n
    5. Set up Amazon A2I to review predictions from Lookout for Metrics.<\/li>\n
    6. Update the model based on output from human loop through Amazon A2I.<\/li>\n<\/ol>\n

      Prerequisites<\/h2>\n

      Before you get started, complete the following steps to set up the Jupyter notebook:<\/p>\n

        \n
      1. Create a notebook instance<\/a> in Amazon SageMaker<\/a>.<\/li>\n
      2. When the notebook is active, choose Open Jupyter<\/strong>.<\/li>\n
      3. On the Jupyter dashboard, choose New<\/strong>, and choose Terminal<\/strong>.<\/li>\n
      4. In the terminal, enter the following code:<\/li>\n<\/ol>\n
        \n
        \n
        git clone \nhttps:\/\/github.com\/aws-samples\/amazon-lookout-for-metrics-a2i-integration.git<\/code><\/pre>\n<\/p><\/div>\n<\/p><\/div>\n
          \n
        1. Open the notebook for this post from next_steps\/A2I_Integration:<\/li>\n<\/ol>\n
          \n
          Amazon_L4M_Detector_A2I.ipynb<\/code><\/pre>\n<\/p><\/div>\n
          You\u2019re now ready to run the notebook cells. Run the environment setup step to set up the necessary Python SDKs and libraries, S3 bucket, and Region that we use throughout the notebook. Make sure that the SageMaker Studio IAM role has the necessary permissions.<\/p>\n
          NOTE: The code uses Python 3.7. Please use the Python 3 (Data Science) kernel for this notebook.<\/p>\n
          Generate synthetic data<\/h2>\n
          In this section, we generate synthetic data for the detector and for predicting anomalies. The metrics data is aggregated on an hourly basis, and the detector runs in continuous mode in real time, every hour. In this example, we use the detector to detect anomalies. Starting from the current date, we generate data for 6 months in the past and 3 days in the future. The historical data is used for training the model, and we use the current and future data for predicting anomalies on an ongoing basis. Note the following configuration:<\/p>\n