An In-depth Walkthrough on Evolution of Neural Machine Translation

TL;DR

This paper aims to study the major trends in Neural Machine Translation, the state of the art models in the domain and a high level comparison between them.

Abstract

Neural Machine Translation (NMT) methodologies have burgeoned from using simple feed-forward architectures to the state of the art; viz. BERT model. The use cases of NMT models have been broadened from just language translations to conversational agents (chatbots), abstractive text summarization, image captioning, etc. which have proved to be a gem in their respective applications. This paper aims to study the major trends in Neural Machine Translation, the state of the art models in the domain and a high level comparison between them.

Figures (10)

• Figure 1: A typical Recurrent Neural Network Cell (with unrolled timesteps)
• Figure 2: Sequence to Sequence Architecture seq2seq
• Figure 3: Additive attention mechanism from bahadanau
• Figure 4: Self Attention
• Figure 5: Convolutional Sequence to Sequence Learning Architecture from convseq