Improving Retrieval-Augmented Neural Machine Translation with Monolingual Data
Maxime Bouthors, Josep Crego, François Yvon
TL;DR
This work investigates leveraging target-language monolingual data to enhance retrieval-augmented NMT by introducing cross-lingual information retrieval (CLIR) that directly retrieves target-side segments. It proposes lexical-aware metric-learning objectives to fine-tune multilingual encoders so that retrieved segments align with lexical overlap, and evaluates three RANMT architectures across multiple language pairs and domains. In controlled settings, CLIR matches TM-based retrieval, and in large-scale monolingual-resource scenarios it delivers substantial translation gains (up to +3.8 BLEU on Wikipedia) with competitive retrieval costs. The findings demonstrate the practical viability of monolingual-target retrieval for improving RANMT and point to future work in scaling encoders, refining CLIR training, and exploring broader domain and language coverage.
Abstract
Conventional retrieval-augmented neural machine translation (RANMT) systems leverage bilingual corpora, e.g., translation memories (TMs). Yet, in many settings, monolingual corpora in the target language are often available. This work explores ways to take advantage of such resources by directly retrieving relevant target language segments, based on a source-side query. For this, we design improved cross-lingual retrieval systems, trained with both sentence level and word-level matching objectives. In our experiments with three RANMT architectures, we assess such cross-lingual objectives in a controlled setting, reaching performances that match those of standard TM-based models. We also showcase our method on a real-world settings, using much larger monolingual and observe strong improvements over both the baseline setting and general-purpose cross-lingual retrievers.