Learning to Parse and Translate Improves Neural Machine Translation

TL;DR

This work addresses the integration of explicit linguistic priors into neural machine translation by proposing NMT+RNNG, a hybrid decoder that jointly learns translation and parsing actions. The model replaces the RNNG buffer with the NMT decoder and uses dual learning signals to train both translation and target-side parsing, while inference can proceed without the external parser. Experiments on four language pairs show BLEU improvements on three pairs and consistent RIBES gains across all, with ablation analyses confirming the value of each component. The approach enhances the linguistic awareness of NMT without increasing inference cost and suggests possibilities for distant supervision to obviate gold parses in the future.

Abstract

There has been relatively little attention to incorporating linguistic prior to neural machine translation. Much of the previous work was further constrained to considering linguistic prior on the source side. In this paper, we propose a hybrid model, called NMT+RNNG, that learns to parse and translate by combining the recurrent neural network grammar into the attention-based neural machine translation. Our approach encourages the neural machine translation model to incorporate linguistic prior during training, and lets it translate on its own afterward. Extensive experiments with four language pairs show the effectiveness of the proposed NMT+RNNG.

Figures (1)

• Figure 1: An example of translation and its dependency relations obtained by our proposed model.