In Nov 2016, Google posted a blog-post about its Google Translate's AI being able to automatically translate between languages, without being explicitly fed with their dictionaries - Lets try to understand it...
In the last 10 years, Google Translate has grown from supporting just a few languages to 103, translating over 140 billion words every day. To make this possible, Google needed to build and maintain many different systems in order to translate between any two languages, incurring significant computational cost.
In Sep 2016 Google Translate switched to a new system called Google Neural Machine Translation (GNMT) - an end-to-end Learning framework that learns from millions of Examples, and provided significant improvements in translation quality. However, while switching to GNMT improved the quality for the languages, scaling up to all the 103 supported languages presented a significant challenge.
Then came - Google’s Multilingual Neural Machine Translation System (GMNMT): Enabling Zero-Shot Translation - Translation between language pairs never seen explicitly by the system, where they addressed the mentioned challenge by extending their previous GNMT system, allowing for a single system to translate between multiple languages.
Here’s how it worked:
Let’s say we train GMNMT with Japanese ⇄ English and Korean ⇄ English examples, shown by the solid blue lines in the animation.
Now, a question:
Can this system translate between a language pair which the system has never seen before?
An example of this would be translations between Korean and Japanese where Korean ⇄ Japanese examples were not shown to the system.
Answer: Yes!!!
It can generate reasonable Korean ⇄ Japanese translations, even though it has never been taught to do so. This is called “zero-shot” translation, shown by the yellow dotted lines in the animation.
But How ???
The system is learning a common representation in which sentences with the same meaning are represented in similar ways regardless of language - i.e. an “interlingua”. Using a 3-dimensional representation of internal network data, we were able to take a peek into the system as it translates a set of sentences between all possible pairs of the Japanese, Korean, and English languages.
Part (a) from the figure above shows an overall geometry of these translations. The points in this view are colored by the meaning; a sentence translated from English to Korean with the same meaning as a sentence translated from Japanese to English share the same color. From this view we can see distinct groupings of points, each with their own color. Part (b) zooms in to one of the groups, and part (c) colors by the source language. Within a single group, we see a sentence with the same meaning but from three different languages. This means the network must be encoding something about the semantics of the sentence rather than simply memorizing phrase-to-phrase translations. We interpret this as a sign of existence of an interlingua in the network.
In the last 10 years, Google Translate has grown from supporting just a few languages to 103, translating over 140 billion words every day. To make this possible, Google needed to build and maintain many different systems in order to translate between any two languages, incurring significant computational cost.
In Sep 2016 Google Translate switched to a new system called Google Neural Machine Translation (GNMT) - an end-to-end Learning framework that learns from millions of Examples, and provided significant improvements in translation quality. However, while switching to GNMT improved the quality for the languages, scaling up to all the 103 supported languages presented a significant challenge.
Then came - Google’s Multilingual Neural Machine Translation System (GMNMT): Enabling Zero-Shot Translation - Translation between language pairs never seen explicitly by the system, where they addressed the mentioned challenge by extending their previous GNMT system, allowing for a single system to translate between multiple languages.
Here’s how it worked:
Now, a question:
Can this system translate between a language pair which the system has never seen before?
An example of this would be translations between Korean and Japanese where Korean ⇄ Japanese examples were not shown to the system.
Answer: Yes!!!
It can generate reasonable Korean ⇄ Japanese translations, even though it has never been taught to do so. This is called “zero-shot” translation, shown by the yellow dotted lines in the animation.
But How ???
The system is learning a common representation in which sentences with the same meaning are represented in similar ways regardless of language - i.e. an “interlingua”. Using a 3-dimensional representation of internal network data, we were able to take a peek into the system as it translates a set of sentences between all possible pairs of the Japanese, Korean, and English languages.
Source:
No comments:
Post a Comment