Disentangling the Roles of Target-Side Transfer and Regularization in Multilingual Machine Translation

TL;DR

This work investigates target-side knowledge transfer and language-induced regularization in one-to-many multilingual MT. By systematically varying auxiliary target languages by linguistic similarity and data size, it shows that similar target languages provide stronger positive transfer while distant targets can regularize the model and improve calibration. The study disentangles transfer from regularization, demonstrates conditions under which each mechanism dominates, and reveals that distant data can boost performance even with minimal transfer. The findings offer practical guidance for selecting auxiliary data to enhance translation quality and reliability in low- and medium-resource settings.

Abstract

Multilingual Machine Translation (MMT) benefits from knowledge transfer across different language pairs. However, improvements in one-to-many translation compared to many-to-one translation are only marginal and sometimes even negligible. This performance discrepancy raises the question of to what extent positive transfer plays a role on the target-side for one-to-many MT. In this paper, we conduct a large-scale study that varies the auxiliary target side languages along two dimensions, i.e., linguistic similarity and corpus size, to show the dynamic impact of knowledge transfer on the main language pairs. We show that linguistically similar auxiliary target languages exhibit strong ability to transfer positive knowledge. With an increasing size of similar target languages, the positive transfer is further enhanced to benefit the main language pairs. Meanwhile, we find distant auxiliary target languages can also unexpectedly benefit main language pairs, even with minimal positive transfer ability. Apart from transfer, we show distant auxiliary target languages can act as a regularizer to benefit translation performance by enhancing the generalization and model inference calibration.

Figures (5)

• Figure 1: The interplay between knowledge transfer and regularization. For one example of main target language Belarusian (language family: Slavic, written script: Cyrillic) the level of knowledge transfer and regularization induced by different auxiliary target languages in MMT.
• Figure 2: Translation quality for En$\rightarrow$De for a low-resource 100K (above), medium-resource 1M (middle) and high-resource 4.5M (below) language pair when training with different auxiliary task numbers and different linguistic groups. Data size represents the total amount of auxiliary target training data.
• Figure 3: Loss curves for En$\rightarrow$De translation tasks under low-resource 100K (a) and medium-resource 1M settings (b), with varying target linguistic groups (similar and distant) and varying auxiliary target data sizes.
• Figure 4: Confidence histograms for En$\rightarrow$De translation tasks under low-resource (100K) (a) and mid-resource (1M) settings (b), with varying target linguistic groups (similar and distant) and total target data sizes.
• Figure 5: Reliability diagrams with inference calibration errors (InfECE) on the En$\rightarrow$De test set in the low-resource (above) and medium-resource setting (below).