EAG: Extract and Generate Multi-way Aligned Corpus for Complete Multi-lingual Neural Machine Translation

TL;DR

The paper tackles the data bottleneck in Complete Multilingual NMT (C-MNMT) by introducing EAG, a two-step method that first extracts candidate alignments from English-centric bilingual corpora using similarity between English sentences via edit distance, and then generates the final aligned targets with a self-supervised, transformer-based generator. This approach yields large-scale, diverse multi-way aligned corpora whose diversity mirrors the original bilingual data, addressing the limited scale of exact matches in prior extraction-based methods and the diversity issues in generation-based pivots. Empirical results on WMT-5 and Opus-100 show that EAG consistently outperforms baselines, including extraction-based C-MNMT and standard MNMT, with BLEU gains up to about 1.1–1.4 points and substantial data-scale advantages, especially for non-English directions. The method thus meaningfully improves translation quality for non-English directions and offers a scalable path to richer multi-way multilingual training data with practical implications for low-resource languages and real-world MT systems.

Abstract

Complete Multi-lingual Neural Machine Translation (C-MNMT) achieves superior performance against the conventional MNMT by constructing multi-way aligned corpus, i.e., aligning bilingual training examples from different language pairs when either their source or target sides are identical. However, since exactly identical sentences from different language pairs are scarce, the power of the multi-way aligned corpus is limited by its scale. To handle this problem, this paper proposes "Extract and Generate" (EAG), a two-step approach to construct large-scale and high-quality multi-way aligned corpus from bilingual data. Specifically, we first extract candidate aligned examples by pairing the bilingual examples from different language pairs with highly similar source or target sentences; and then generate the final aligned examples from the candidates with a well-trained generation model. With this two-step pipeline, EAG can construct a large-scale and multi-way aligned corpus whose diversity is almost identical to the original bilingual corpus. Experiments on two publicly available datasets i.e., WMT-5 and OPUS-100, show that the proposed method achieves significant improvements over strong baselines, with +1.1 and +1.4 BLEU points improvements on the two datasets respectively.

Figures (3)

• Figure 1: Examples constructed by EAG. "$x^1_i\leftrightarrow y^1_i$" and "$x^2_j \leftrightarrow y^2_j$" represent the bilingual examples in English $\rightarrow$ Arabic and English $\rightarrow$ Chinese respectively. $\tilde{y}^2_j$ is the generated Chinese sentence, which is aligned to $x^1_i$ and $y^1_i$. For a clear presentation, the Google translation (in English) for the generated $\tilde{y}_j^2$ is also provided.
• Figure 2: Experimental results on testing the hyper-parameters. With $\gamma$ set as 0, this is the setting of the baseline system of freitag2020complete.
• Figure 3: Examples constructed by EAG. "$x^1$-$y^1$" and "$x^2$-$y^2$" represent the bilingual examples in English $\rightarrow$ Arabic and English $\rightarrow$ Chinese respectively. $\tilde{y}^2$ is the generated Chinese sentence. The words in red color are the different word compositions between $x^1$ and $x^2$. The Google translations (in English) for $\tilde{y}^2$ are also provided.