{"id":1137,"date":"2021-11-03T08:40:18","date_gmt":"2021-11-03T08:40:18","guid":{"rendered":"https:\/\/salarydistribution.com\/machine-learning\/2021\/11\/03\/intelligently-split-multi-form-document-packages-with-amazon-textract-and-amazon-comprehend\/"},"modified":"2021-11-03T08:40:18","modified_gmt":"2021-11-03T08:40:18","slug":"intelligently-split-multi-form-document-packages-with-amazon-textract-and-amazon-comprehend","status":"publish","type":"post","link":"https:\/\/salarydistribution.com\/machine-learning\/2021\/11\/03\/intelligently-split-multi-form-document-packages-with-amazon-textract-and-amazon-comprehend\/","title":{"rendered":"Intelligently split multi-form document packages with Amazon Textract and Amazon Comprehend"},"content":{"rendered":"
\n

Many organizations spanning different sizes and industry verticals still rely on large volumes of documents to run their day-to-day operations. To solve this business challenge, customers are using intelligent document processing services from AWS such as Amazon Textract<\/a> and Amazon Comprehend<\/a> to help with extraction and process automation<\/a>. Before you can extract text, key-value pairs, tables, and entities, you need to be able to split multipage PDF documents that often contain heterogeneous form types. For example, in mortgage processing, a broker or loan processing individual may need to split a consolidated PDF loan package, containing the mortgage application (Fannie Mae form 1003), W2s, income verification, 1040 tax forms, and more.<\/p>\n

To tackle this problem, organizations use rules-based processing: identifying document types via form titles, page numbers, form lengths, and so on. These approaches are error-prone and difficult to scale, especially when the form types may have several variations. Accordingly, these workarounds break down quickly in practice and increase the need for human intervention.<\/p>\n

In this post, we show how you can create your own document splitting solution with little code for any set of forms, without building custom rules or processing workflows.<\/p>\n

Solution overview<\/h2>\n

For this post, we use a set of common mortgage application forms to demonstrate how you can use Amazon Textract and Amazon Comprehend to create an intelligent document splitter that is more robust than earlier approaches. When processing documents for mortgage applications, the borrower submits a multipage PDF that is made up of heterogeneous document types of varying page lengths; to extract information, the user (for example, a bank) must break down this PDF.<\/p>\n

Although we show a specific example for mortgage forms, you can generally scale and apply this approach to just about any set of multi-page PDF documents.<\/p>\n

We use Amazon Textract to extract data from the document and build an Amazon Comprehend compatible dataset to train a document classification model<\/a>. Next, we train the classification model and create a classification endpoint that can perform real-time document analysis. Keep in mind that Amazon Textract and Amazon Comprehend classification endpoints incur charges, so refer to Amazon Textract pricing<\/a> and Amazon Comprehend pricing<\/a> for more information. Finally, we show how we can classify documents with this endpoint and split documents based on the classification results.<\/p>\n

This solution uses the following AWS services:<\/p>\n

Prerequisites<\/h2>\n

You need to complete the following prerequisites to build and deploy this solution:<\/p>\n

    \n
  1. Install<\/a> Python 3.8.x.<\/li>\n
  2. Install<\/a> jq.<\/li>\n
  3. Install<\/a> the AWS SAM CLI.<\/li>\n
  4. Install Docker<\/a>.<\/li>\n
  5. Make sure you have pip installed<\/a>.<\/li>\n
  6. Install and configure<\/a> the AWS Command Line Interface<\/a> (AWS CLI).<\/li>\n
  7. Configure<\/a> your AWS credentials.<\/li>\n<\/ol>\n

    The solution is designed to work optimally in the us-east-1<\/code> and us-west-2<\/code> Regions to take advantage of higher default quotas for Amazon Textract. For specific Regional workloads, refer to Amazon Textract endpoints and quotas<\/a>. Make sure you use a single Region for the entire solution.<\/p>\n

    Clone the repo<\/h2>\n

    To get started, clone the repository by running the following command; then we switch into the working directory:<\/p>\n

    \n
    git clone https:\/\/github.com\/aws-samples\/aws-document-classifier-and-splitter.git\ncd aws-document-classifier-and-splitter<\/code><\/pre>\n<\/p><\/div>\n
    Solution workflows<\/h2>\n
    The solution consists of three workflows:<\/p>\n