{"id":1343,"date":"2021-12-11T01:24:07","date_gmt":"2021-12-11T01:24:07","guid":{"rendered":"https:\/\/salarydistribution.com\/machine-learning\/2021\/12\/11\/live-transcriptions-of-f1-races-using-amazon-transcribe\/"},"modified":"2021-12-11T01:24:07","modified_gmt":"2021-12-11T01:24:07","slug":"live-transcriptions-of-f1-races-using-amazon-transcribe","status":"publish","type":"post","link":"https:\/\/salarydistribution.com\/machine-learning\/2021\/12\/11\/live-transcriptions-of-f1-races-using-amazon-transcribe\/","title":{"rendered":"Live transcriptions of F1 races using Amazon Transcribe"},"content":{"rendered":"
\n

The Formula 1 (F1) live steaming service, F1 TV, has live automated closed captions in three different languages: English, Spanish, and French.<\/p>\n

For the 2021 season, FORMULA 1 has achieved another technological breakthrough, building a fully automated workflow to create closed captions in three languages and broadcasting to 85 territories using Amazon Transcribe<\/a>. Amazon Transcribe is an automatic speech recognition (ASR) service that allows you to generate audio transcription.<\/p>\n

In this post, we share how Formula 1 joined forces with the AWS Professional Services<\/a> team to make it happen. We discuss how they used Amazon Transcribe and its custom vocabulary feature as well as custom-built postprocessing logic to improve their live transcription accuracy in three languages.<\/p>\n

The challenge<\/h2>\n

For F1, everything is about extreme speed: with pit stops as short as 2 seconds, speeds of up to 375 KPH (233 MPH), and 5g forces on drivers under braking and through corners. In this fast-paced and dynamic environment, milliseconds dictate the difference between pole position or second on the grid. The role of the race commentators is to weave the multitude of parallel events and information into a single exciting narrative. This form of commentary greatly increases the engagement and excitement of viewers.<\/p>\n

F1 has a strong affinity to cutting edge technology, and partnered with AWS to build a scalable and sustainable closed caption solution for F1 TV, their Over-the-top (OTT) platform, that can support a growing calendar and language portfolio. F1 now provides real-time live captions in three languages across four series: F1 in British English, US Spanish and French; and F2, F3, and Porsche Supercup in British English and US Spanish. This was achieved using Amazon Transcribe to automatically convert the commentary into subtitles.<\/p>\n

This task provides many unique challenges. With the excitement of an F1 race, it\u2019s common to have commentators with differing accents move quickly from one topic to another as the race unfolds. Being a sport steeped in technology, commentators often refer to F1 domain-specific terminology such as DRS (Drag Reduction System), aerodynamic, downforce, or halo (a safety device) for example. Moreover, F1 is a global sport, traveling across the world and drawing drivers from many different countries. Looking only at the 2021 season, 16\/20 drivers had non-English names and 17\/20 had non-Spanish names or non-French names. With the advanced customization features available in Amazon Transcribe, we tailored the underlying language models to recognize domain-specific terms that are rare in general language use, which boosted transcription accuracy.<\/p>\n

In the following sections, we take a deep dive into how AWS Professional Services partnered with F1 to build a robust, state-of-the-art, real-time race commentary captioning system by enhancing Amazon Transcribe to understand the particularities of the F1 world. You will learn how to utilize Amazon Transcribe in real-time broadcasts and supercharge live captioning for your use case with custom vocabularies, postprocessing steps, and a human-in-the-loop validation layer.<\/p>\n

Solution overview<\/h2>\n

The solution works as a proxy to Amazon Transcribe. Custom vocabularies are passed as parameters to Amazon Transcribe, and the resulting captions are postprocessed. The postprocessed text is then moderated by an F1 moderator before being transformed to captions that are displayed to the viewers. The following diagram shows the sequential process.<\/p>\n

\"\"<\/p>\n

Live transcriptions: Understanding use case specific terminology and context<\/h2>\n

The output of Automatic Speech Recognition (ASR) systems is highly context-dependent. ASR language models benefit from utilizing the words across a fully spoken sentence. For example, in the following sentence, the system uses the words \u2018WORLD CHAMPIONSHIP\u2019 towards the end of the sentence to inform context and allow \u2018FORMER ONE\u2019 to be correctly transcribed as \u2018FORMULA 1\u2019.<\/p>\n\n\n\n
\n

GOOD AFTERNOON EVERYBODY WELCOME ALONG TO ROUND 4 OF THE FORMER ONE<\/strong><\/p>\n

\u00a0<\/strong><\/p>\n

GOOD AFTERNOON EVERYBODY WELCOME ALONG TO ROUND 4 OF THE FORMULA 1 <\/strong>WORLD CHAMPIONSHIP IN 2019<\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

Amazon Transcribe supports both batch <\/em>and streaming <\/em>transcription models. In batch transcription, the model issues a transcription using the full context provided in the audio segment. Amazon Transcribe streaming transcription enables you to send an audio stream and receive a transcription stream in real time. Generating subtitles for a live broadcast requires a streaming model because transcriptions should appear on screen shortly after the commentary is spoken. This real-time need presents unique challenges compared to batch transcriptions and often affects the quality of the results because the language model has limited knowledge of the future context.<\/p>\n

Amazon Transcribe is pre-trained to capture a wide range of use cases. However, F1 domain-specific terminology, names, and locations aren\u2019t present in the Amazon Transcribe general language model. Getting those words correct is nevertheless crucial for the understanding of the narrative, such as who is leading the race, circuit corners, and technical details.<\/p>\n

Amazon Transcribe allows you to develop with custom vocabularies and custom language models to improve transcription accuracy. You can use them separately for streaming transcriptions or together for batch transcriptions.<\/p>\n

Custom vocabularies<\/a> consist of a list of specific words that you want Amazon Transcribe to recognize in the audio input. These are generally domain-specific words and phrases, such as proper nouns. You can inform Amazon Transcribe how to pronounce these terms with information such as SoundsLike (in regular orthography) or the IPA (International Phonetic Alphabet<\/a>) description of the term. Custom vocabularies are available for all languages supported by Amazon Transcribe<\/a>. Custom vocabularies improve the ability of Amazon Transcribe to recognize terms without using the context in which they\u2019re spoken.<\/p>\n

The following table shows some examples of a custom vocabulary.<\/p>\n\n\n\n\n\n\n\n\n\n\n\n
Phrase<\/strong><\/td>\nDisplayAs<\/strong><\/td>\nSoundsLike<\/strong><\/td>\nIPA<\/strong><\/td>\n<\/tr>\n
Charles-Leclerc<\/td>\nCharles Leclerc<\/td>\n<\/td>\n\u0283 \u0251 \u0279 l l \u0259 k l \u025b \u0279<\/td>\n<\/tr>\n
Charles-Leclerc<\/td>\nCharles Leclerc<\/td>\nshal-luh-klurk<\/td>\n<\/td>\n<\/tr>\n
Lewis-Hamilton<\/td>\nLewis Hamilton<\/td>\nloo-is-ha-muhl-tn<\/td>\n<\/td>\n<\/tr>\n
Lewis-Hamilton<\/td>\nLewis Hamilton<\/td>\nloo-uhs-ha-muhl-tn<\/td>\n<\/td>\n<\/tr>\n
Ferrari<\/td>\nFerrari<\/td>\n<\/td>\nf \u025d \u0279 \u0251 \u0279 \u026a<\/td>\n<\/tr>\n
Ferrari<\/td>\nFerrari<\/td>\nfuh-rehr-ee<\/td>\n<\/td>\n<\/tr>\n
Mercedes<\/td>\nMercedes<\/td>\nmer-sey-deez<\/td>\n<\/td>\n<\/tr>\n
Mercedes<\/td>\nMercedes<\/td>\n<\/td>\nm \u025b \u0279 s e\u026a d i z<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

The custom vocabulary includes the following details:<\/p>\n