Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

PMMT: Preference Alignment in Multilingual Machine Translation via LLM Distillation

Shuqiao Sun, Yutong Yao, Peiwen Wu, Feijun Jiang, Kaifu Zhang

TL;DR

Experiments indicate that the proposed method takes the lead in translation tasks with aligned human preferences by a large margin, and shows a competitive performance compared to SOTA works.

Abstract

Translation is important for cross-language communication, and many efforts have been made to improve its accuracy. However, less investment is conducted in aligning translations with human preferences, such as translation tones or styles. In this paper, a new method is proposed to effectively generate large-scale multilingual parallel corpora with specific translation preferences using Large Language Models (LLMs). Meanwhile, an automatic pipeline is designed to distill human preferences into smaller Machine Translation (MT) models for efficiently and economically supporting large-scale calls in online services. Experiments indicate that the proposed method takes the lead in translation tasks with aligned human preferences by a large margin. Meanwhile, on popular public benchmarks like WMT and Flores, on which our models were not trained, the proposed method also shows a competitive performance compared to SOTA works.

PMMT: Preference Alignment in Multilingual Machine Translation via LLM Distillation

TL;DR

Experiments indicate that the proposed method takes the lead in translation tasks with aligned human preferences by a large margin, and shows a competitive performance compared to SOTA works.

Abstract

Translation is important for cross-language communication, and many efforts have been made to improve its accuracy. However, less investment is conducted in aligning translations with human preferences, such as translation tones or styles. In this paper, a new method is proposed to effectively generate large-scale multilingual parallel corpora with specific translation preferences using Large Language Models (LLMs). Meanwhile, an automatic pipeline is designed to distill human preferences into smaller Machine Translation (MT) models for efficiently and economically supporting large-scale calls in online services. Experiments indicate that the proposed method takes the lead in translation tasks with aligned human preferences by a large margin. Meanwhile, on popular public benchmarks like WMT and Flores, on which our models were not trained, the proposed method also shows a competitive performance compared to SOTA works.

Paper Structure

This paper contains 33 sections, 5 equations, 4 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Translation Models
  4. Dataset Generation
  5. Preference Alignment
  6. Approach
  7. Seed Dataset
  8. Length.
  9. Language.
  10. Danger words.
  11. Number.
  12. Similarity.
  13. PMMT-P Model
  14. PMMT-J Model
  15. Correctness.
  16. ...and 18 more sections

Figures (4)

  • Figure 1: Pipeline of the proposed PMMT method.
  • Figure 2: Dataset distribution. Gray blocks: no need for translation. Blue blocks: have data; White blocks: no data. G1/G3: az, bn, cs, el, fa, fi, hi, hu, id, ko, ms, my, ne, nl, pt, ro, th, uk. G2/G4: fr, ja, pl, vi.
  • Figure 3: Detailed comparisons between different translation methods.
  • Figure 4: Accuracy of the PMMT-J model trained with different data and model scales.