Promoting Target Data in Context-aware Neural Machine Translation

TL;DR

This work investigates promoting target-language context within a standard concatenation-based context-aware NMT framework. By prepending target context to the source (with or without source context) and using simple contextual concatenation, the authors achieve large improvements on target-oriented phenomena across Basque–Spanish and English–Russian, with the strongest gains when both contexts are combined on the source side ($ ext{src+tgt-}n ext{to}n$). The study confirms robustness to back-translated data and shows only modest impact from using machine-translated context at inference, suggesting practical benefits without architectural changes. Overall, targeting target-context information significantly enhances discourse-aware translation and offers a viable, architecture-agnostic path for document-level MT.

Abstract

Standard context-aware neural machine translation (NMT) typically relies on parallel document-level data, exploiting both source and target contexts. Concatenation-based approaches in particular, still a strong baseline for document-level NMT, prepend source and/or target context sentences to the sentences to be translated, with model variants that exploit equal amounts of source and target data on each side achieving state-of-the-art results. In this work, we investigate whether target data should be further promoted within standard concatenation-based approaches, as most document-level phenomena rely on information that is present on the target language side. We evaluate novel concatenation-based variants where the target context is prepended to the source language, either in isolation or in combination with the source context. Experimental results in English-Russian and Basque-Spanish show that including target context in the source leads to large improvements on target language phenomena. On source-dependent phenomena, using only target language context in the source achieves parity with state-of-the-art concatenation approaches, or slightly underperforms, whereas combining source and target context on the source side leads to significant gains across the board.