Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Ensemble Self-Training for Unsupervised Machine Translation

Ido Aharon, Jonathan Shaki, Sarit Kraus

Abstract

We present an ensemble-driven self-training framework for unsupervised neural machine translation (UNMT). Starting from a primary language pair, we train multiple UNMT models that share the same translation task but differ in an auxiliary language, inducing structured diversity across models. We then generate pseudo-translations for the primary pair using token-level ensemble decoding, averaging model predictions in both directions. These ensemble outputs are used as synthetic parallel data to further train each model, allowing the models to improve via shared supervision. At deployment time, we select a single model by validation performance, preserving single-model inference cost. Experiments show statistically significant improvements over single-model UNMT baselines, with mean gains of 1.7 chrF when translating from English and 0.67 chrF when translating into English.

Ensemble Self-Training for Unsupervised Machine Translation

Abstract

We present an ensemble-driven self-training framework for unsupervised neural machine translation (UNMT). Starting from a primary language pair, we train multiple UNMT models that share the same translation task but differ in an auxiliary language, inducing structured diversity across models. We then generate pseudo-translations for the primary pair using token-level ensemble decoding, averaging model predictions in both directions. These ensemble outputs are used as synthetic parallel data to further train each model, allowing the models to improve via shared supervision. At deployment time, we select a single model by validation performance, preserving single-model inference cost. Experiments show statistically significant improvements over single-model UNMT baselines, with mean gains of 1.7 chrF when translating from English and 0.67 chrF when translating into English.
Paper Structure (46 sections, 8 equations, 6 figures, 10 tables, 1 algorithm)

This paper contains 46 sections, 8 equations, 6 figures, 10 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. UNMT with Decoder-Only LLMs
  4. Ensemble Learning
  5. Ensemble Methods for Machine Translation
  6. Evaluation
  7. Method
  8. Overview
  9. Problem Setup
  10. Base Model Training
  11. Monolingual Training
  12. Mixed Training
  13. Pure Back-Translation
  14. Ensemble-Driven Self-Training
  15. Ensemble Decoding
  16. ...and 31 more sections

Figures (6)

  • Figure 1: 100 sentences BLEU for Llama-3.2-1B: mean performance across all single models.
  • Figure 2: Llama-3.2-1B performance during ensemble training (10 models).
  • Figure 3: Qwen3-0.6B-Base performance during ensemble training (10 models).
  • Figure 4: Qwen3-1.7B-Base performance during ensemble training (10 models).
  • Figure 5: 100 sentences BLEU for Qwen3-0.6B-Base: mean performance across all single models.
  • ...and 1 more figures