{"id":1171,"date":"2021-11-09T08:34:47","date_gmt":"2021-11-09T08:34:47","guid":{"rendered":"https:\/\/salarydistribution.com\/machine-learning\/2021\/11\/09\/augment-search-with-metadata-by-chaining-amazon-textract-amazon-comprehend-and-amazon-kendra\/"},"modified":"2021-11-09T08:34:47","modified_gmt":"2021-11-09T08:34:47","slug":"augment-search-with-metadata-by-chaining-amazon-textract-amazon-comprehend-and-amazon-kendra","status":"publish","type":"post","link":"https:\/\/salarydistribution.com\/machine-learning\/2021\/11\/09\/augment-search-with-metadata-by-chaining-amazon-textract-amazon-comprehend-and-amazon-kendra\/","title":{"rendered":"Augment search with metadata by chaining Amazon Textract, Amazon Comprehend, and Amazon Kendra"},"content":{"rendered":"
\n

Amazon Kendra<\/a> is an intelligent search service powered by machine learning (ML). Amazon Kendra reimagines enterprise search for your websites and applications so your employees and customers can easily find the content they\u2019re looking for, even when it\u2019s scattered across multiple locations and content repositories within your organization. With Amazon Kendra, you can stop searching through troves of unstructured data and discover the right answers to your questions, when you need them.<\/p>\n

Although Amazon Kendra is a great search tool, it only performs as well as the quality of documents in its index. Like with all things AI\/ML related, the better the quality of data that is input into Amazon Kendra, the more targeted and precise the search results. So how can we improve the documents in our Amazon Kendra index to maximize search result performance? To allow Amazon Kendra to return more targeted search results, we enrich the documents with metadata to use attributes such as main language, named entities, key phrases, and more.<\/p>\n

In this post, we address the following use case: With large amounts of raw historical documents to search on, how do we connect metadata to the documents to take advantage of Amazon Kendra\u2019s boosting and filtering features? We aim to demonstrate a way in which you can enrich your historical data by adding metadata to searchable documents with Amazon Textract and Amazon Comprehend, to get more targeted and flexible searches with Amazon Kendra.<\/p>\n

Note that the following tutorial is written using Amazon SageMaker Notebooks as a code platform. However, these API calls can be done using any IDE of your choice. To save costs, and for the sake of your own familiarity, feel free to use your favorite IDE in place of SageMaker Notebooks to follow along.<\/p>\n

Solution overview<\/h2>\n

For this post, we examine a hypothetical use case for a media entertainment company. We have many documents about movies and television shows and want to use Amazon Kendra to query the data. For demonstration purposes, we pull public data on Wikipedia<\/a> to create PDF documents that act as our company\u2019s data that we want to query on. We use an Amazon SageMaker<\/a> notebook instance<\/a> as our code platform. We use Python, along with the Boto3 Python library<\/a>, to connect to and use the Amazon Textract<\/a>, Amazon Comprehend<\/a>, and Amazon Kendra APIs.<\/p>\n

We walk you through the following high-level steps:<\/p>\n

    \n
  1. Create our media PDF documents through Wikipedia.<\/li>\n
  2. Create metadata using Amazon Textract and Amazon Comprehend.<\/li>\n
  3. Configure the Amazon Kendra index and load the data.<\/li>\n
  4. Run a sample query and experiment with boosting query performance.<\/li>\n<\/ol>\n

    Prerequisites<\/h2>\n

    As a prerequisite, we first set up a SageMaker notebook instance and a Python notebook within it.<\/p>\n

    Create a SageMaker notebook instance<\/h3>\n

    To create a SageMaker notebook instance, you can follow the instructions in the documentation Create a Notebook Instance<\/a>, or follow the configuration that we use in this post.<\/p>\n

      \n
    1. Create a notebook instance with the following configuration:\n
        \n
      1. Notebook instance name<\/strong> \u2013 KendraAugmentation<\/code><\/li>\n
      2. Notebook instance class <\/strong>\u2013 ml.t2.medium<\/li>\n
      3. Elastic inference <\/strong>\u2013 None<\/li>\n<\/ol>\n<\/li>\n<\/ol>\n

        Next, we create an AWS Identity and Access Management<\/a> (IAM) role.<\/p>\n

          \n
        1. Choose Create a new role<\/strong>.<\/li>\n
        2. Choose Next<\/strong> to create a role.<\/li>\n<\/ol>\n

          The role name starts with AmazonSageMaker-ExecutionRole-xxxxxxxx<\/code>. For this example, we create a role called AmazonSageMaker-ExecutionRole-Kendra-Blog<\/strong><\/code>.<\/p>\n

            \n
          1. For Root access<\/strong>, select Enable<\/strong>.<\/li>\n
          2. Leave the remaining options at their default.<\/li>\n
          3. Choose Create notebook instance<\/strong>.<\/li>\n<\/ol>\n

            You\u2019re redirected to a page that shows that your notebook instance is being created. The process takes a few minutes. When you see a green InService<\/strong><\/code> state, the notebook is ready.<\/p>\n

            Create a Python3 notebook in your SageMaker notebook instance<\/h3>\n

            When your SageMaker instance is ready, choose the version of Jupyter you prefer to use. For this post, we use the original Jupyter notebook as opposed to JupyterLab.
            \"\"<\/a><\/p>\n

            When inside, create a new conda_python3<\/strong><\/code> notebook.
            \"\"<\/a><\/p>\n

            With this, we\u2019re ready to start writing and running Python code. To run the rest of the code that follows, run the following code in the first Jupyter notebook cell to import the necessary modules we need:<\/p>\n

            \n
            # Module Imports\nimport boto3\nimport os<\/code><\/pre>\n<\/p><\/div>\n
            As we go through each of the sections, we import other modules as necessary.<\/p>\n
            Create Media PDF documents through Wikipedia<\/h2>\n
            Run the following code in one of the Jupyter notebook cells to create an Amazon Simple Storage Service<\/a> (Amazon S3) bucket where we store all the media documents that we search on:<\/p>\n
            \n
            # Instantiate Amazon S3 client.\ns3_client = boto3.client('s3')\n\n# Create bucket.\nbucket_name = \"kendra-augmentation-documents-jp\"\ns3_client.create_bucket(Bucket=bucket_name)\n\n# List buckets to make sure bucket was created.\nresponse = s3_client.list_buckets()\nprint('Existing buckets:')\nfor bucket in response['Buckets']:\n    print(f'  {bucket[\"Name\"]}')<\/code><\/pre>\n<\/p><\/div>\n
            For this post, our bucket is kendra-augmentation-documents-jp<\/strong><\/code>. You can update the code with a different name.<\/p>\n
            As we mentioned earlier, we create mock PDF documents from public Wikipedia content that represent the media data that we augment and perform searches on. I\u2019ve pre-selected movies and TV shows from the entertainment industry in the following code, but you can choose different topics in your notebook.<\/p>\n
            \n
            # Import fpdf module.\nfrom fpdf import FPDF\n\n# Movie and TV show topics for media documents.\ntopics = [\"Dumb & Dumber Movie\", \"Black Panther Movie\",\n          \"Star Wars The Last Jedi Movie\", \"Mary Poppins Movie\",\n          \"Kung Fu Panda Movie\", \"I Love Lucy\", \"The Office TV Show\",\n          \"Star Trek: The Original Series\", \"NCIS TV Show\",\n          \"Game of Thrones TV Show\"]\n\n# Create PDF documents out of each topic and store into Amazon S3 bucket.\nfor topic in topics:\n    \n    # Define text and PDF document names.\n    text_filename = f\"{topic}.txt\".replace(\" \",\"_\")\n    pdf_filename = text_filename.replace(\"txt\",\"pdf\")\n    \n    # Write to text first.\n    with open(text_filename, \"w+\") as f:\n        f.write(wikipedia.summary(topic))\n        \n    # Convert text to pdf.\n    pdf = FPDF()\n    with open(text_filename, 'rb') as text_file:\n        txt = text_file.read().decode('latin-1')\n    pdf.set_font('Times', '', 12)\n    pdf.add_page()\n    pdf.multi_cell(0, 5, txt)\n    pdf.output(pdf_filename, 'F')\n    \n    # Upload to Amazon S3 bucket.\n    s3_client = boto3.client('s3')\n    s3_client.upload_file(pdf_filename,\n                          \"kendra-augmentation-documents-jp\",\n                          pdf_filename)\n    os.remove(text_filename)\n    os.remove(pdf_filename)<\/code><\/pre>\n<\/p><\/div>\n
            When this code block finishes running, we have 10 media PDF documents that we can augment with metadata using Amazon Textract and Amazon Comprehend, then run queries on with Amazon Kendra.
            \"\"<\/a><\/p>\n
            Create metadata using Amazon Textract and Amazon Comprehend<\/h2>\n
            To create metadata for each of our PDF files, we must first extract the text portions of each PDF using Amazon Textract. We run the extracted text through Amazon Comprehend to attach attributes (metadata) to the PDFs, such as named entities, dominant language, and key phrases. Note that Amazon Comprehend will be able to read PDF files directly in a future feature release.<\/p>\n
              \n
            1. Use the following helper function (s3_get_filenames<\/strong><\/code>) to get all the file names in a specific bucket or prefix folder in Amazon S3:<\/li>\n<\/ol>\n
              \n
              def s3_get_filenames(bucket, prefix=None):\n    \"\"\"\n    Gets all the filenames in a specific bucket\/prefix folder in Amazon S3.\n    \n    Parameters:\n    ----------\n    bucket : str\n        String representing bucket name you want to get filenames from.\n    prefix : str\n        String representing prefix within the bucket that you want to get\n        filenames from.\n        \n    Returns:\n    -------\n    file_list : list[str]\n        List containing all filenames within the bucket\/prefix location\n    \"\"\"\n    \n    # Set Amazon S3 client and get file objects.\n    s3 = boto3.client('s3')\n    if prefix == None:\n        prefix = ''\n    result = s3.list_objects(Bucket=bucket, Prefix=prefix)\n    \n    # Put all file names into one list.\n    file_list = []\n    for obj in result['Contents']:\n        # Only take objects that are not the folder.\n        if 'metadata\/' not in obj['Key']:\n            file_list += [obj['Key']]\n        \n    return file_list<\/code><\/pre>\n<\/p><\/div>\n
              We run Amazon Textract on each of our PDF files to extract the text of each file and transform the data into a format that we later ingest into Amazon Comprehend.<\/p>\n
              Next, we create the S3 bucket and service role settings needed to run Amazon Textract through SageMaker notebook instances.<\/p>\n
                \n
              1. Create a new S3 bucket to store our Amazon Textract output, called kendra-augmentation-textract-output-jp<\/strong><\/code>:<\/li>\n<\/ol>\n
                \n
                # Create a new bucket for Amazon Textract text outputs:\nbucket_name = \"kendra-augmentation-textract-output-jp\"\ns3_client.create_bucket(Bucket=bucket_name)<\/code><\/pre>\n<\/p><\/div>\n
                  \n
                1. Attach the AmazonTextractFullAccess<\/strong><\/code> policy to the same AmazonSageMaker-ExecutionRole-Kendra-Blog <\/strong><\/code>role.<\/li>\n<\/ol>\n
                  This policy allows SageMaker to access Amazon Textract.
                  \"\"<\/a><\/p>\n
                    \n
                  1. Run the following code to run Amazon Textract on our PDF files, create new .txt files for Amazon Comprehend to use, and send these files to the S3 bucket we created:<\/li>\n<\/ol>\n
                    \n
                    # Amazon Textract specific imports.\n!pip install amazon-textract-caller amazon-textract-prettyprinter\nfrom textractcaller.t_call import call_textract, Textract_Features\nfrom textractprettyprinter.t_pretty_print import Textract_Pretty_Print, get_lines_string\nimport textractprettyprinter.t_pretty_print as prettyprint\n\n# Transform and output documents from s3.\ninput_bucket = 'kendra-augmentation-documents-jp'\noutput_bucket = 'kendra-augmentation-textract-output-jp'\ns3 = boto3.client('s3')\n\n# Get all the names of the media documents we want to run Textract on.\ninput_documents = s3_get_filenames(input_bucket)\n\n# Loop through input documents, transform textract outputs into LINE\n# transformations, and output to S3 for ingestion into Comprehend.\nfor document_name in input_documents:\n    \n    # Define input document to read.\n    input_document = f's3:\/\/{input_bucket}\/{document_name}'\n    \n    # Get text using Textract call_textract function.\n    textract_json = call_textract(input_document=input_document)\n    \n    # Convert response from Textract using get_lines_string function.\n    line_transformation_text = get_lines_string(textract_json=textract_json)\n\n    # Put text into text file to send back to S3.\n    filename = f'{document_name}_LINE.txt'\n    with open(filename,'w+') as f:\n        f.write(line_transformation_text)\n        \n    # Send text file to S3 to be ingested into Comprehend.\n    with open(filename, 'rb') as data:\n        s3.upload_fileobj(data, output_bucket, filename)<\/code><\/pre>\n<\/p><\/div>\n
                    We now have a .txt Amazon Textract output file for each of our PDFs in the kendra-augmentation-textract-output-jp<\/strong><\/code> bucket.
                    \"\"<\/a><\/p>\n
                    Now that we have our .txt files with the text representation of our PDF files, we can create metadata out of them using Amazon Comprehend.<\/p>\n
                      \n
                    1. Attach the ComprehendFullAccess<\/strong><\/code> policy to the AmazonSageMaker-ExecutionRole-Kendra-Blog<\/strong><\/code> role.
                      \"\"<\/a><\/li>\n<\/ol>\n
                      We extract and attach the following Amazon Comprehend metadata attributes to each document for Amazon Kendra to index on:<\/p>\n