{"id":450,"date":"2020-10-27T00:32:34","date_gmt":"2020-10-27T00:32:34","guid":{"rendered":"https:\/\/machine-learning.webcloning.com\/2020\/10\/27\/building-an-nlu-powered-search-application-with-amazon-sagemaker-and-the-amazon-es-knn-feature\/"},"modified":"2020-10-27T00:32:34","modified_gmt":"2020-10-27T00:32:34","slug":"building-an-nlu-powered-search-application-with-amazon-sagemaker-and-the-amazon-es-knn-feature","status":"publish","type":"post","link":"https:\/\/salarydistribution.com\/machine-learning\/2020\/10\/27\/building-an-nlu-powered-search-application-with-amazon-sagemaker-and-the-amazon-es-knn-feature\/","title":{"rendered":"Building an NLU-powered search application with Amazon SageMaker and the Amazon ES KNN feature"},"content":{"rendered":"
\n

The rise of semantic search engines has made ecommerce and retail businesses search easier for its consumers. Search engines powered by natural language understanding<\/a> (NLU) allow you to speak or type into a device using your preferred conversational language rather than finding the right keywords for fetching the best results. You can query using words or sentences in your native language, leaving it to the search engine to deliver the best results.<\/p>\n

Amazon SageMaker<\/a> is a fully managed service that provides every developer and data scientist with the ability to build, train, and deploy machine learning (ML) models quickly. Amazon Elasticsearch Service<\/a> (Amazon ES) is a fully managed service that makes it easy for you to deploy, secure, and run Elasticsearch cost-effectively at scale. Amazon ES offers KNN search, which can enhance search in use cases such as product recommendations, fraud detection, and image, video, and some specific semantic scenarios like document and query similarity. Alternatively, you can also choose Amazon Kendra<\/a>, a highly accurate and easy to use enterprise search service that\u2019s powered by machine learning, with no machine learning experience required. In this post, we explain how you can implement an NLU-based product search for certain types of applications using Amazon SageMaker\u00a0and the Amazon ES k-nearest neighbor (KNN) feature.<\/p>\n

In the post Building a visual search application with Amazon SageMaker and Amazon ES<\/a>,<\/u> we shared how to build a visual search application using Amazon SageMaker and the Amazon ES KNN\u2019s Euclidean distance<\/a> metric. Amazon ES now supports open-source Elasticsearch version 7.7 and includes the cosine similarity<\/a> metric for KNN indexes. Cosine similarity measures the cosine of the angle between two vectors in the same direction, where a smaller cosine angle denotes higher similarity between the vectors. With cosine similarity, you can measure the orientation between two vectors, which makes it the ideal choice for some specific semantic search applications. The highly distributed architecture of Amazon ES enables you to implement an enterprise-grade search engine with enhanced KNN ranking, with high recall and performance.<\/p>\n

In this post, you build a very simple search application that demonstrates the potential of using KNN with Amazon ES compared to the traditional Amazon ES ranking method, including a web application for testing the KNN-based search queries in your browser. The application also compares the search results with Elasticsearch match queries to demonstrate the difference between KNN search and full-text search.<\/p>\n

Overview of solution<\/h2>\n

Regular Elasticsearch text-matching search is useful when you want to do text-based search, but KNN-based search is a more natural way to search for something. For example, when you search for a wedding dress using KNN-based search application, it gives you similar results if you type \u201cwedding dress\u201d or \u201cmarriage dress.\u201d Implementing this KNN-based search application consists of two phases:<\/p>\n