{"id":612,"date":"2020-11-24T08:04:40","date_gmt":"2020-11-24T08:04:40","guid":{"rendered":"https:\/\/machine-learning.webcloning.com\/2020\/11\/24\/customizing-your-machine-translation-using-amazon-translate-active-custom-translation\/"},"modified":"2020-11-24T08:04:40","modified_gmt":"2020-11-24T08:04:40","slug":"customizing-your-machine-translation-using-amazon-translate-active-custom-translation","status":"publish","type":"post","link":"https:\/\/salarydistribution.com\/machine-learning\/2020\/11\/24\/customizing-your-machine-translation-using-amazon-translate-active-custom-translation\/","title":{"rendered":"Customizing your machine translation using Amazon Translate Active Custom Translation"},"content":{"rendered":"
\n

When translating the English phrase \u201cHow are you?\u201d to Spanish, would you prefer to use \u201c\u00bfC\u00f3mo est\u00e1s?\u201d or \u201c\u00bfC\u00f3mo est\u00e1 usted?\u201d instead?<\/p>\n

Amazon Translate<\/a> is a neural machine translation service that delivers fast, high-quality, and affordable language translation. Today, we\u2019re excited to introduce Active Custom Translation (ACT), a feature that gives you more control over your machine translation output. You can now influence what machine translation output you would like to get between \u201c\u00bfC\u00f3mo est\u00e1s?\u201d or \u201c\u00bfC\u00f3mo est\u00e1 usted?\u201d. To make ACT work, simply provide your translation examples in TMX, TSV, or CSV format to create parallel data (PD), and Amazon Translate uses your PD along with your batch translation job<\/a> to customize the translation output at runtime. If you have PD that shows \u201cHow are you?\u201d being translated to \u201c\u00bfC\u00f3mo est\u00e1 usted?\u201d, ACT knows to customize the translation to \u201c\u00bfC\u00f3mo est\u00e1 usted?\u201d.<\/p>\n

Today, professional translators use examples of previous translations to provide more customized translations for customers. Similar to profession translators, Amazon Translate can now provide customized translations by learning from your translation examples.<\/p>\n

Traditionally, this customization was done by creating a custom translation model\u00ad\u2014a specific-purpose translation engine built using customer data. Building custom translation models is complex, tedious, and expensive. It requires special expertise to prepare the data for training, testing, and validation. Then you build, deploy, and maintain the model by updating the model frequently. To save on model training and management costs, you may choose to delay updating your custom translation model, which means your models are always stale\u2014negatively affecting your custom translation experience. In spite of all this work, these custom models perform well when the translation job is within the domain of your data. However, they tend to perform worse than a generic model when the translation job is outside of the domain of your customization data.<\/p>\n

Amazon Translate ACT introduces an innovative way of providing customized translation output on the fly with your parallel data, without building a custom translation model. ACT output quality is always up to date with your PD. ACT provides the best translations for jobs both within the domain and outside the domain of PD. For example, if a source sentence isn\u2019t in the domain of the PD, the translation output is still as good as the generic translation with no significant deterioration in translation quality. You no longer need to go through the tedious process of building and retraining custom translation models for each incoming use case. Just update the PD, and the ACT output automatically adapts to the most recent PD, without needing any retraining.<\/p>\n

\u201cInnovation is in our DNA. Our customers look to AWS to lead in customization of machine translation. Current custom translation technology is inefficient, cumbersome, and expensive,\u201d says Marcello Federico, Principal Applied Scientist at Amazon Machine Learning, AWS. \u201cActive Custom Translation allows our customers to focus on the value of their latest data and forget about the lifecycle management of custom translation models. We innovated on behalf of the customer to make custom machine translation easy.\u201d<\/p>\n

Don\u2019t just take our word for it<\/h2>\n

Custom.MT<\/a> implements machine translation for localization groups and translation companies. Konstantin Dranch, Custom.MT co-founder, shares, \u201cAmazon Translate\u2019s ACT is a breakthrough machine translation setup. A manual engine retraining takes 15\u201316 work hours, that\u2019s why most language teams in the industry update their engines only once a month or once a quarter. With ACT, retraining is continuous and engines improve every day based on edits by human translators. Even before the feature was released to the market, we saw tremendous interest from leading software localization teams. With a higher quality of machine translation, enterprise teams can save millions of USD in manual translations and improve other KPIs, such as international user engagement and time to market.\u201d<\/p>\n

Welocalize<\/a> is a leading global localization and translation company. Senior Manager of AI Deployments at Welocalize Alex Yanishevsky says, \u201cWelocalize produces high-quality translations, so our customers can transform their content and data to grow globally and expand into international markets. Active Custom Translation from Amazon Translate allows us to customize our translations at runtime and provides us with significant flexibility in our production cycles. In addition, we see great business value and engine quality improvement since we can retrain engines frequently without incurring additional hosting or training charges.\u201d<\/p>\n

One Hour Translation<\/a> is a leading professional language services provider. Yair Tal, CEO of One Hour Translation, says, \u201cThe customer demand for customized Neural Machine Translation (NMT) is growing every month because of the cost savings. As one of the first to try Amazon Translate ACT, we have found that ACT provides the best translation output for many language pairs. With ACT, training and maintenance is simple and the Translate API integrates with our system seamlessly. Translate\u2019s pay-as-you-translate pricing helps our clients, both big and small, get translation output that is tailored for their needs without paying to train custom models.\u201d<\/p>\n

Building an Active Custom Translation job<\/h2>\n

Active Custom Translation\u2019s capabilities are built right into the Amazon Translate experience. In this post, we walk you through the step-by-step process of using your data and getting a customized machine translated output securely. ACT is now available on batch translation, so first familiarize yourself with how to create a batch translation<\/a> job.<\/p>\n

You need data to customize your translation for terms or phrases that are unique to a specific domain, such as life sciences, law, or finance. You bring in examples of high-quality translations (source sentence and translated target sentence) in your preferred domain as a file in TMX<\/a>, TSV<\/a>, or CSV<\/a> format. This data should also be UTF-8<\/a> encoded. You use this data to create a PD. Amazon Translate uses this PD to customize your machine translation. Each PD can be up to 1 GB large. You can upload up to 1,000 PD per account per Region. The 1,000 parallel data limit can be increased upon request. You get free storage for parallel data for up to 200 GB. You pay the local Amazon Simple Storage Service<\/a> (Amazon S3) rate for excess data stored.<\/p>\n

For our use case, I have my data in TSV format, and the name of my file is Mydata.tsv<\/code>. I first upload this file to an S3 location (for this post, I store my data in s3:\/\/input-s3bucket\/Paralleldata\/<\/code>).<\/p>\n

The following table summarizes the contents of the file.<\/p>\n\n\n\n\n\n\n
en<\/strong><\/td>\nes<\/strong><\/td>\n<\/tr>\n
Amazon Translate is a neural machine translation service.<\/td>\nAmazon Translate es un servicio de traducci\u00f3n autom\u00e1tica basado en redes neuronales.<\/td>\n<\/tr>\n
Neural machine translation is a form of language translation automation that uses deep learning models.<\/td>\nLa traducci\u00f3n autom\u00e1tica neuronal es una forma de automatizar la traducci\u00f3n de lenguajes utilizando modelos de aprendizaje profundo.<\/td>\n<\/tr>\n
How are you?<\/td>\n\u00bfC\u00f3mo est\u00e1 usted?<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

We run this example in the US West (Oregon) Region, us-west-2<\/code>.<\/p>\n

CreateParallelData<\/h3>\n

Calling the CreateParallelData<\/code> API creates a PD resource record in our database and asynchronously starts a workflow for processing the PD file and ingesting it into our service.<\/p>\n

CLI<\/h4>\n

The following CLI commands are formatted for Unix, Linux, and macOS. For Windows, replace the backslash () Unix continuation character at the end of each line with a caret (^).<\/p>\n

Run the following CLI command:<\/p>\n

\n
aws translate create-parallel-data \r\n--name ${PARALLEL_DATA_NAME}\r\n--parallel-data-config S3Uri=${S3_URI},Format=${FORMAT} \r\n--region ${REGION}<\/code><\/pre>\n<\/div>\n
I use Mydata.tsv<\/code> to create my PD my-parallel-data-1<\/code>:<\/p>\n
\n
aws translate create-parallel-data \r\n--name my-parallel-data-1 \r\n--parallel-data-config S3Uri= s3:\/\/input-s3bucket\/Paralleldata\/Mydata.tsv,Format=TSV \r\n--region us-west-2 <\/code><\/pre>\n<\/div>\n
You get a response like the following code:<\/p>\n
\n
{\r\n    \"Name\": \"my-parallel-data-1\",\r\n    \"Status\": \"CREATING\"\r\n}<\/code><\/pre>\n<\/div>\n
This means that your PD is being created now.<\/p>\n
Run aws translate create-parallel-data<\/code> help for more information.<\/p>\n
Console<\/h4>\n
To use the Amazon Translate console, complete the following steps:<\/p>\n
    \n
  1. On the Amazon Translate console, under Customization<\/strong>, choose Parallel data<\/strong>.<\/li>\n
  2. Choose Create parallel data<\/strong>.<\/li>\n<\/ol>\n
    \"\"<\/p>\n
      \n
    1. For Name<\/strong>, insert my-parallel-data-1<\/code>.<\/li>\n
    2. For Parallel data location in S3<\/strong>, enter your S3 location (for this post, s3:\/\/input-s3bucket\/Paralleldata\/Mydata.tsv<\/code>).<\/li>\n
    3. For File format<\/strong>\u00b8 you can choose CSV, TSV, or TMX. For this post, we choose Tab-separated values (.tsv)<\/strong>.<\/li>\n<\/ol>\n
      Your data is always secure with Amazon Translate. It\u2019s encrypted using an AWS owned encryption key<\/a> by default. You can encrypt it using a key from your current account or use a key from a different account.<\/p>\n
        \n
      1. For this post, for Encryption key<\/strong>, we select Use AWS owned key<\/strong>.<\/li>\n
      2. Choose Create parallel data<\/strong>.<\/li>\n<\/ol>\n
        \"\"<\/p>\n
        ListParallelData<\/h3>\n
        Calling the ListParallelData<\/code> API returns a list of PD that exists and their details (it doesn\u2019t include a pre-signed Amazon S3 URL for downloading the data)<\/p>\n
        CLI<\/h4>\n
        Run the following CLI command:<\/p>\n
        \n
        aws translate list-parallel-data \r\n--region us-west-2<\/code><\/pre>\n<\/div>\n
        You get a response like the following code:<\/p>\n
        \n
        {\r\n    \"ParallelDataPropertiesList\": [\r\n        {\r\n            \"Name\": \"my-parallel-data-1\",\r\n            \"Arn\": \"arn:aws:translate:us-west-2:123456789012:parallel-data\/my-parallel-data-1\",\r\n            \"Status\": \"ACTIVE\",\r\n            \"SourceLanguageCode\": \"en\",\r\n            \"TargetLanguageCodes\": [\r\n                \"es\"\r\n            ],\r\n            \"ParallelDataConfig\": {\r\n                \"S3Uri\": \"s3:\/\/input-s3bucket\/Paralleldata\/Mydata.tsv\",\r\n                \"Format\": \"TSV\"\r\n            },\r\n            \"ImportedDataSize\": 532,\r\n            \"ImportedRecordCount\": 3,\r\n            \"FailedRecordCount\": 0,\r\n            \"CreatedAt\": 1234567890.406,\r\n            \"LastUpdatedAt\": 1234567890.675\r\n        }\r\n    ]\r\n}<\/code><\/pre>\n<\/div>\n
        The \"Status\": \"ACTIVE\"<\/code> means your PD is ready for you to use.<\/p>\n
        Run aws translate list-parallel-data help<\/code> for more information.<\/p>\n
        Console<\/h4>\n
        This following screenshot shows the result for list-parallel-data on the Amazon Translate console.<\/p>\n
        \"\"<\/p>\n
        GetParallelData<\/h3>\n
        Calling the GetParallelData<\/code> API returns details of the named parallel data and a pre-signed Amazon S3 URL for downloading the data.<\/p>\n
        CLI<\/h4>\n
        Run the following CLI command:<\/p>\n
        \n
        aws translate get-parallel-data \r\n--name ${PARALLEL_DATA_NAME} \r\n--region ${REGION}<\/code><\/pre>\n<\/div>\n
        For example, my code looks like the following:<\/p>\n
        \n
        aws translate get-parallel-data \r\n--name my-parallel-data-1 \r\n--region us-west-2<\/code><\/pre>\n<\/div>\n
        You get a response like the following code:<\/p>\n
        \n
        {\r\n    \"ParallelDataProperties\": {\r\n        \"Name\": \"my-parallel-data-1\",\r\n        \"Arn\": \"arn:aws:translate:us-west-2:123456789012:parallel-data\/my-parallel-data-1\",\r\n        \"Status\": \"ACTIVE\",\r\n        \"SourceLanguageCode\": \"en\",\r\n        \"TargetLanguageCodes\": [\r\n            \"es\"\r\n        ],\r\n        \"ParallelDataConfig\": {\r\n            \"S3Uri\": \"s3:\/\/input-s3bucket\/Paralleldata\/Mydata.tsv\",\r\n            \"Format\": \"TSV\"\r\n        },\r\n        \"ImportedDataSize\": 532,\r\n        \"ImportedRecordCount\": 3,\r\n        \"FailedRecordCount\": 0,\r\n        \"CreatedAt\": 1234567890.406,\r\n        \"LastUpdatedAt\": 1234567890.675\r\n    },\r\n    \"DataLocation\": {\r\n        \"RepositoryType\": \"S3\",\r\n        \"Location\": \"xxx\"\r\n    }\r\n}<\/code><\/pre>\n<\/div>\n
        \u201cLocation\u201d<\/code> contains the pre-signed Amazon S3 URL for downloading the data.<\/p>\n
        Run aws translate get-parallel-data help<\/code> for more information.<\/p>\n
        Console<\/h4>\n
        On the Amazon Translate console, choose one of the PD files on the Parallel data <\/strong>page.<\/p>\n
        \"\"<\/p>\n
        You\u2019re directed to another page that includes the detail for this parallel data file. The following screenshot shows the details for get-parallel-data<\/code>.<\/p>\n
        \"\"<\/p>\n
        UpdateParallelData<\/h3>\n
        Calling the UpdateParallelData<\/code> API replaces the old parallel data with the new one.<\/p>\n
        CLI<\/h4>\n
        Run the following CLI command:<\/p>\n
        \n
        aws translate update-parallel-data \r\n--name ${PARALLEL_DATA_NAME}\r\n--parallel-data-config S3Uri=${NEW_S3_URI},Format=${FORMAT} \r\n--region us-west-2<\/code><\/pre>\n<\/div>\n
        For this post, Mydata1.tsv<\/code> is my new parallel data. My code looks like the following:<\/p>\n
        \n
        aws translate update-parallel-data \r\n--name my-parallel-data-1 \r\n--parallel-data-config S3Uri= s3:\/\/input-s3bucket\/Paralleldata\/Mydata1.tsv,Format=TSV \r\n--region us-west-2<\/code><\/pre>\n<\/div>\n
        You get a response like the following code:<\/p>\n
        \n
        {\r\n    \"Name\": \"my-parallel-data-1\",\r\n    \"Status\": \"ACTIVE\",\r\n    \"LatestUpdateAttemptStatus\": \"UPDATING\",\r\n    \"LatestUpdateAttemptAt\": 1234567890.844\r\n}<\/code><\/pre>\n<\/div>\n
        The \"LatestUpdateAttemptStatus\": \"UPDATING\"<\/code> means your parallel data is being updated now.<\/p>\n
        Wait for a few minutes and run get-parallel-data again. You can see the parallel data get updated, such as in the following code:<\/p>\n
        \n
        {\r\n    \"ParallelDataProperties\": {\r\n            \"Name\": \"my-parallel-data-1\",\r\n            \"Arn\": \"arn:aws:translate:us-west-2:123456789012:parallel-data\/my-parallel-data-1\",\r\n            \"Status\": \"ACTIVE\",\r\n            \"SourceLanguageCode\": \"en\",\r\n            \"TargetLanguageCodes\": [\r\n                \"es\"\r\n            ],\r\n            \"ParallelDataConfig\": {\r\n                \"S3Uri\": \"s3:\/\/input-s3bucket\/Paralleldata\/Mydata1.tsv\",\r\n                \"Format\": \"TSV\"\r\n            },\r\n        ...\r\n    }\r\n}<\/code><\/pre>\n<\/div>\n
        We can see that the parallel data has been updated from Mydata.tsv<\/code> to Mydata1.tsv<\/code>.<\/p>\n
        Run aws translate update-parallel-data help<\/code> for more information.<\/p>\n
        Console<\/h4>\n
        On the Amazon Translate console, choose the parallel data file and choose Update<\/strong>.<\/p>\n
        \"\"<\/p>\n
        You can replace the new parallel data file with the existing one by specifying the new Amazon S3 URL.<\/p>\n
        \"\"<\/p>\n
        Creating your first Active Custom Translation job<\/h2>\n
        In this section, we discuss the different ways you can create your ACT job.<\/p>\n
        StartTextTranslationJob<\/h3>\n
        Calling the StartTextTranslationJob<\/code> starts a batch translation. When you add parallel data to a batch translation job, you create an ACT job. Amazon Translate customizes your ACT output to match the style, tone, and word choices it finds in your PD. ACT is a premium product, so see Amazon Translate pricing<\/a> for pricing information. You can only specify one parallel data file to use with the text translation job.<\/p>\n
        CLI<\/h4>\n
        Run the following command:<\/p>\n
        \n
        aws translate start-text-translation-job \r\n--input-data-config ContentType=${CONTENT_TYPE},S3Uri=${INPUT_S3_URI} \r\n--output-data-config S3Uri=${OUTPUT_S3_URI} \r\n--data-access-role-arn ${DATA_ACCESS_ROLE}\r\n--source-language-code=${SOURCE_LANGUAGE_CODE} --target-language-codes=${TARGET_LANGUAGE_CODE} \r\n--parallel-data-names ${PARALLEL_DATA_NAME}\r\n--region ${REGION}\r\n--job-name ${JOB_NAME}<\/code><\/pre>\n<\/div>\n
        For example, my code looks like the following:<\/p>\n
        \n
        aws translate start-text-translation-job \r\n--input-data-config ContentType=application\/vnd.openxmlformats-officedocument.spreadsheetml.sheet,S3Uri= s3:\/\/input-s3bucket\/inputfile\/ \r\n--output-data-config S3Uri= s3:\/\/output-s3bucket\/Output\/ \r\n--data-access-role-arn arn:aws:iam::123456789012:role\/TranslateBatchAPI \r\n--source-language-code=en --target-language-codes=es \r\n--parallel-data-names my-parallel-data-1 \r\n--region us-west-2 \r\n--job-name ACT1<\/code><\/pre>\n<\/div>\n
        You get a response like the following code:<\/p>\n
        \n
        {\r\n    \"JobId\": \"4446f95f20c88a4b347449d3671fbe3d\",\r\n    \"JobStatus\": \"SUBMITTED\"\r\n}<\/code><\/pre>\n<\/div>\n
        This output means the job has been submitted successfully.<\/p>\n
        Run aws translate start-text-translation-job<\/code> help for more information.<\/p>\n
        Console<\/h4>\n
        For instructions on running a batch translation job on the Amazon Translate console, see Translating documents, spreadsheets, and presentations in Office Open XML format using Amazon Translate<\/a>. Choose my-parallel-data-1<\/code> as the parallel data to create your first ACT job, ACT1.<\/p>\n
        \"\"<\/p>\n
        Congratulations! You have created your first ACT job. ACT is available in the following Regions:<\/p>\n