Revisiting Context Choices for Context-aware Machine Translation

TL;DR

It is shown that multi-source transformer models improve MT over standard transformer-base models even with empty lines provided as context, but the translation quality improves significantly when a sufficient amount of correct context is provided.

Abstract

One of the most popular methods for context-aware machine translation (MT) is to use separate encoders for the source sentence and context as multiple sources for one target sentence. Recent work has cast doubt on whether these models actually learn useful signals from the context or are improvements in automatic evaluation metrics just a side-effect. We show that multi-source transformer models improve MT over standard transformer-base models even with empty lines provided as context, but the translation quality improves significantly (1.51 - 2.65 BLEU) when a sufficient amount of correct context is provided. We also show that even though randomly shuffling in-domain context can also improve over baselines, the correct context further improves translation quality and random out-of-domain context further degrades it.

Figures (2)

• Figure 1: Distribution of the position of the antecedent from the current sentence in Japanese and English. 0 means the antecedent of a anaphora is in the same sentence, 1 means the previous sentence and so on. For Japanese corpora (Kyoto University Text Corpus and Japanese Web Corpus), the antecedent of the omitted element (zero-anaphora) are investigated. For English corpora (OntoNotes 5.0 Broadcast Conversation and Telephone Conversations), the antecedent of all coreference relations are investigated.
• Figure 2: Best EN$\leftrightarrow$JA results compared to the baseline and 0 context, random out-of-domain (ood) context, and random in-domain (ind) context.