“…All that changed in September, when Google gave their translation tool a new engine: the Google Neural Machine Translation system (GNMT). This new engine comes fully loaded with all the hot 2016 buzzwords, like neural network and machine learning…”
So it was already the new engine that had Thomas Mann writing about “the ice-cream marching around” and “the sieves of science rattling in the seals of the sea” while “now and then, with pious eyes, the hats were torn down in front of the Wotanshut and the Jupiter’s beard of a headmaster who was measured”?
The should’ve stuck with the old system maybe? :-0
This seems to be the key point in the article:
"Here’s how it works. Let’s say we train a multilingual system with Japanese⇄English and Korean⇄English examples, shown by the solid blue lines in the animation. Our multilingual system, with the same size as a single GNMT system, shares its parameters to translate between these four different language pairs. This sharing enables the system to transfer the “translation knowledge” from one language pair to the others. This transfer learning and the need to translate between multiple languages forces the system to better use its modeling power.
This inspired us to ask the following question: Can we translate between a language pair which the system has never seen before? An example of this would be translations between Korean and Japanese where Korean⇄Japanese examples were not shown to the system. Impressively, the answer is yes — it can generate reasonable Korean⇄Japanese translations, even though it has never been taught to do so. We call this “zero-shot” translation, shown by the yellow dotted lines in the animation. To the best of our knowledge, this is the first time this type of transfer learning has worked in Machine Translation."
Andrew, thanks for this. It looks important.
The Google Translate guys say they are running 103 languages and translating 106 billion words each year, so obviously they are cutting edge and way ahead of anyone else in the field.
I suppose that all 103 languages are basically doing the same thing - i.e. communicating knowledge of the outside world to other human brains using the organs of the human body as the transmission vehicle. So the patterns in all 103 languages must be basically the same.
It seems that when they put in language pairs for some languages, the network is then using them to make its own language model which is able to make pairs in new languages.
I don’t know very much about AI. Are we talking about emergent properties of complex systems here? i…e. once the neural network gets complex enough it starts to develop new abilities, as perhaps human consciousness came into existence once the brain had reached a certain level of complexity?
However, I am way out of my depth here!
PS a friend yesterday pointed out to me an article in the Economist Jan 11 2017 called “Machines Learn to Speak Human Language” where they appear be using a “new language model”. Not sure if this is the same topic.
Some folks take this seriously - indeed deadly seriously.
Fascinating! Thank you for sharing. I certainly missed this, although it matches many of my main interests: languages, data science, machine learning, Psychology, …
I found this seemingly in-depth explanation of how “Zero-shot translation” works:
No it doesn’t. All computers are programmed.
Look up “emergent properties”.