Exploring Parameter-Efficient Fine-Tuning and Backtranslation for the WMT 25 General Translation Task
Felipe Fujita, Hideyuki Takada
TL;DR
This work addresses English→Japanese neural machine translation under limited data by combining backtranslation with parameter-efficient fine-tuning (LoRA) of a strong decoder (Mistral 7B). The approach augments a small Japanese–English parallel corpus with synthetic data and adapts the model to translation-domain signals, achieving COMET scores of $0.589$ with FT alone and $0.597$ with FT+BT, outperforming BT or FT in isolation. The findings demonstrate that data augmentation via backtranslation provides complementary diversity while targeted fine-tuning drives the bulk of quality gains, offering a practical blueprint for low-resource Japanese translation under constrained resources. This lightweight, dataset-efficient strategy has practical implications for researchers and practitioners working with limited corpora and compute.
Abstract
In this paper, we explore the effectiveness of combining fine-tuning and backtranslation on a small Japanese corpus for neural machine translation. Starting from a baseline English{\textrightarrow}Japanese model (COMET = 0.460), we first apply backtranslation (BT) using synthetic data generated from monolingual Japanese corpora, yielding a modest increase (COMET = 0.468). Next, we fine-tune (FT) the model on a genuine small parallel dataset drawn from diverse Japanese news and literary corpora, achieving a substantial jump to COMET = 0.589 when using Mistral 7B. Finally, we integrate both backtranslation and fine-tuning{ -- }first augmenting the small dataset with BT generated examples, then adapting via FT{ -- }which further boosts performance to COMET = 0.597. These results demonstrate that, even with limited training data, the synergistic use of backtranslation and targeted fine-tuning on Japanese corpora can significantly enhance translation quality, outperforming each technique in isolation. This approach offers a lightweight yet powerful strategy for improving low-resource language pairs.