Adversarial Neural Machine Translation

TL;DR

The paper introduces Adversarial-NMT, a framework that replaces standard likelihood-based training with an adversarial objective where a CNN discriminator distinguishes human vs. machine translations. The NMT generator is trained via REINFORCE to produce translations that fool the discriminator, with MLE included for stability. Empirical results on En→Fr and De→En show BLEU gains and positive human judgments, demonstrating the viability of adversarial training for sequence-to-sequence translation. This approach advances beyond traditional MLE-based training by optimizing a sequence-level, human-aligned objective and sets a new benchmark for translation quality in monitored setups.

Abstract

In this paper, we study a new learning paradigm for Neural Machine Translation (NMT). Instead of maximizing the likelihood of the human translation as in previous works, we minimize the distinction between human translation and the translation given by an NMT model. To achieve this goal, inspired by the recent success of generative adversarial networks (GANs), we employ an adversarial training architecture and name it as Adversarial-NMT. In Adversarial-NMT, the training of the NMT model is assisted by an adversary, which is an elaborately designed Convolutional Neural Network (CNN). The goal of the adversary is to differentiate the translation result generated by the NMT model from that by human. The goal of the NMT model is to produce high quality translations so as to cheat the adversary. A policy gradient method is leveraged to co-train the NMT model and the adversary. Experimental results on English$\rightarrow$French and German$\rightarrow$English translation tasks show that Adversarial-NMT can achieve significantly better translation quality than several strong baselines.

Figures (3)

• Figure 1: The Adversarial-NMT framework. 'Ref' is short for 'Reference' which means the ground-truth translation and 'Hyp' is short for 'Hypothesis', denoting model translation sentence. All the yellow parts denote the NMT model $G$, which maps a source sentence $x$ to a translation sentence. The red parts are the adversary network $D$, which predicts whether a given target sentence is the ground-truth translation of the given source sentence $x$. $G$ and $D$ combat with each other, generating both sampled translation $y'$ to train $D$, and the reward signals to train $G$ by policy gradient (the blue arrows).
• Figure 2: The CNN adversary framework.
• Figure 3: Dev set BLEUs during En$\rightarrow$Fr Adversarial-NMT training process, with same learning rates for $D$, different learning rates for $G$ in left \ref{['fig:ana_fixD']}, and same learning rates for $G$ and different learning rates for $D$ in right \ref{['fig:ana_fixG']}.