Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Improving Low-Resource Machine Translation via Round-Trip Reinforcement Learning

Ahmed Attia, Alham Fikri

TL;DR

This work investigates a self-supervised reinforcement-learning-based fine-tuning for translation in low-resource settings using round-trip bootstrapping with the No Language Left Behind (NLLB) family of models, and argues that this method can further benefit from scale, enabling models to increasingly leverage their pretrained knowledge and continue self-improving.

Abstract

Low-resource machine translation (MT) has gained increasing attention as parallel data from low-resource language communities is collected, but many potential methods for improving low-resource MT remain unexplored. We investigate a self-supervised reinforcement-learning-based fine-tuning for translation in low-resource settings using round-trip bootstrapping with the No Language Left Behind (NLLB) family of models. Our approach translates English into a target low-resource language and then back into English, using a combination of chrF++ and BLEU as the reward function on the reconstructed English sentences. Using the NLLB-MD dataset, we evaluate both the 600M and 1.3B parameter NLLB models and observe consistent improvements for the following languages: Central Aymara, Friulian, Wolof and Russian. Qualitative inspection of translation outputs indicates increased fluency and semantic fidelity. We argue that our method can further benefit from scale, enabling models to increasingly leverage their pretrained knowledge and continue self-improving.

Improving Low-Resource Machine Translation via Round-Trip Reinforcement Learning

TL;DR

This work investigates a self-supervised reinforcement-learning-based fine-tuning for translation in low-resource settings using round-trip bootstrapping with the No Language Left Behind (NLLB) family of models, and argues that this method can further benefit from scale, enabling models to increasingly leverage their pretrained knowledge and continue self-improving.

Abstract

Low-resource machine translation (MT) has gained increasing attention as parallel data from low-resource language communities is collected, but many potential methods for improving low-resource MT remain unexplored. We investigate a self-supervised reinforcement-learning-based fine-tuning for translation in low-resource settings using round-trip bootstrapping with the No Language Left Behind (NLLB) family of models. Our approach translates English into a target low-resource language and then back into English, using a combination of chrF++ and BLEU as the reward function on the reconstructed English sentences. Using the NLLB-MD dataset, we evaluate both the 600M and 1.3B parameter NLLB models and observe consistent improvements for the following languages: Central Aymara, Friulian, Wolof and Russian. Qualitative inspection of translation outputs indicates increased fluency and semantic fidelity. We argue that our method can further benefit from scale, enabling models to increasingly leverage their pretrained knowledge and continue self-improving.
Paper Structure (39 sections, 6 equations, 5 figures, 5 tables, 1 algorithm)

This paper contains 39 sections, 6 equations, 5 figures, 5 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Improving NMT with Monolingual Data and Roundtrip Translations
  4. Reinforcement Learning and GRPO for NMT
  5. Method
  6. Preliminary
  7. Problem Formulation
  8. Round-Trip Translation Reinforcement Training
  9. Phase 1: Forward Translation.
  10. Phase 2: Back-Translation.
  11. Reward Design
  12. chrF++.
  13. BLEU.
  14. Experiments
  15. Results
  16. ...and 24 more sections

Figures (5)

  • Figure 1: Overview Of Our Low-Resource Machine Translation With Round-Trip Reinforcement Learning Method.
  • Figure 2: forward translations chrF++ gain (after -- before) for different reward functions across model/language settings.
  • Figure 3: Validation curves showing forward translation chrF++ scores for the four languages after 8K optimization steps.
  • Figure 4: Training curves showing the English backward translation scores during training for the 1.3B and 600M parameter models on the 4 Languages.
  • Figure 5: Validation curves after 8K optimization steps. Top row: forward translation chrF++ scores. Bottom row: backward translation BLEU scores.