Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Relay Decoding: Concatenating Large Language Models for Machine Translation

Chengpeng Fu, Xiaocheng Feng, Yichong Huang, Wenshuai Huo, Baohang Li, Hui Wang, Bin Qin, Ting Liu

TL;DR

An innovative approach called RD (Relay Decoding), which entails concatenating two distinct large models that individually support the source and target languages, achieves superior results in the machine translation task.

Abstract

Leveraging large language models for machine translation has demonstrated promising results. However, it does require the large language models to possess the capability of handling both the source and target languages in machine translation. When it is challenging to find large models that support the desired languages, resorting to continuous learning methods becomes a costly endeavor. To mitigate these expenses, we propose an innovative approach called RD (Relay Decoding), which entails concatenating two distinct large models that individually support the source and target languages. By incorporating a simple mapping layer to facilitate the connection between these two models and utilizing a limited amount of parallel data for training, we successfully achieve superior results in the machine translation task. Experimental results conducted on the Multi30k and WikiMatrix datasets validate the effectiveness of our proposed method.

Relay Decoding: Concatenating Large Language Models for Machine Translation

TL;DR

An innovative approach called RD (Relay Decoding), which entails concatenating two distinct large models that individually support the source and target languages, achieves superior results in the machine translation task.

Abstract

Leveraging large language models for machine translation has demonstrated promising results. However, it does require the large language models to possess the capability of handling both the source and target languages in machine translation. When it is challenging to find large models that support the desired languages, resorting to continuous learning methods becomes a costly endeavor. To mitigate these expenses, we propose an innovative approach called RD (Relay Decoding), which entails concatenating two distinct large models that individually support the source and target languages. By incorporating a simple mapping layer to facilitate the connection between these two models and utilizing a limited amount of parallel data for training, we successfully achieve superior results in the machine translation task. Experimental results conducted on the Multi30k and WikiMatrix datasets validate the effectiveness of our proposed method.
Paper Structure (20 sections, 3 equations, 3 figures, 4 tables)

This paper contains 20 sections, 3 equations, 3 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Approach
  3. Task Description
  4. Concatenate Method
  5. Training Strategy
  6. Experiment
  7. Experimental Details
  8. Large Language Models
  9. Datasets
  10. Baselines
  11. Main Results
  12. Analysis
  13. Is it necessary to finetune the LLMs during training?
  14. How much data is required to complete the training of the mapping layer?
  15. Related Work
  16. ...and 5 more sections

Figures (3)

  • Figure 1: LLaMA's supported languages include English and French while Aquila mainly support English and Chinese. Both Aquila2 and LLaMA are not proficient in handling the Chinese to French translation task individually. In such cases, we can concatenate the two models to accomplish the translation task.
  • Figure 2: Using Chinese-French translation as a case in point for the process of Relay Decoding.
  • Figure 3: Different mapping layers.