Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LLMs Beyond English: Scaling the Multilingual Capability of LLMs with Cross-Lingual Feedback

Wen Lai, Mohsen Mesgar, Alexander Fraser

TL;DR

This work tackles the challenge of scaling LLM multilinguality to 100 languages while aligning outputs with human preferences. It introduces xLLMs-100, trained via a multilingual instruction dataset and a cross-lingual feedback dataset using LoRA-based fine-tuning and DPO alignment, respectively. The model demonstrates state-of-the-art performance across five multilingual benchmarks, improving both understanding and generation and reducing language-off-target issues. The study also analyzes the impact of training data choices on language democratization and identifies avenues for expanding languages and addressing remaining biases and safety concerns.

Abstract

To democratize large language models (LLMs) to most natural languages, it is imperative to make these models capable of understanding and generating texts in many languages, in particular low-resource ones. While recent multilingual LLMs demonstrate remarkable performance in such capabilities, these LLMs still support a limited number of human languages due to the lack of training data for low-resource languages. Moreover, these LLMs are not yet aligned with human preference for downstream tasks, which is crucial for the success of LLMs in English. In this paper, we introduce xLLaMA-100 and xBLOOM-100 (collectively xLLMs-100), which scale the multilingual capabilities of LLaMA and BLOOM to 100 languages. To do so, we construct two datasets: a multilingual instruction dataset including 100 languages, which represents the largest language coverage to date, and a cross-lingual human feedback dataset encompassing 30 languages. We perform multilingual instruction tuning on the constructed instruction data and further align the LLMs with human feedback using the DPO algorithm on our cross-lingual human feedback dataset. We evaluate the multilingual understanding and generating capabilities of xLLMs-100 on five multilingual benchmarks. Experimental results show that xLLMs-100 consistently outperforms its peers across the benchmarks by considerable margins, defining a new state-of-the-art multilingual LLM that supports 100 languages.

LLMs Beyond English: Scaling the Multilingual Capability of LLMs with Cross-Lingual Feedback

TL;DR

This work tackles the challenge of scaling LLM multilinguality to 100 languages while aligning outputs with human preferences. It introduces xLLMs-100, trained via a multilingual instruction dataset and a cross-lingual feedback dataset using LoRA-based fine-tuning and DPO alignment, respectively. The model demonstrates state-of-the-art performance across five multilingual benchmarks, improving both understanding and generation and reducing language-off-target issues. The study also analyzes the impact of training data choices on language democratization and identifies avenues for expanding languages and addressing remaining biases and safety concerns.

Abstract

To democratize large language models (LLMs) to most natural languages, it is imperative to make these models capable of understanding and generating texts in many languages, in particular low-resource ones. While recent multilingual LLMs demonstrate remarkable performance in such capabilities, these LLMs still support a limited number of human languages due to the lack of training data for low-resource languages. Moreover, these LLMs are not yet aligned with human preference for downstream tasks, which is crucial for the success of LLMs in English. In this paper, we introduce xLLaMA-100 and xBLOOM-100 (collectively xLLMs-100), which scale the multilingual capabilities of LLaMA and BLOOM to 100 languages. To do so, we construct two datasets: a multilingual instruction dataset including 100 languages, which represents the largest language coverage to date, and a cross-lingual human feedback dataset encompassing 30 languages. We perform multilingual instruction tuning on the constructed instruction data and further align the LLMs with human feedback using the DPO algorithm on our cross-lingual human feedback dataset. We evaluate the multilingual understanding and generating capabilities of xLLMs-100 on five multilingual benchmarks. Experimental results show that xLLMs-100 consistently outperforms its peers across the benchmarks by considerable margins, defining a new state-of-the-art multilingual LLM that supports 100 languages.
Paper Structure (35 sections, 1 figure, 27 tables)

This paper contains 35 sections, 1 figure, 27 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Multilingual Capabilities of LLMs.
  4. Aligning LLMs with Human Feedback.
  5. Method
  6. Multilingual Instruction Dataset
  7. Instruction Translation.
  8. Hybrid Response Generation.
  9. Cross-Lingual Feedback Dataset
  10. Instruction Design.
  11. Response Generation.
  12. Instruction and Response Quality
  13. Multilingual Instruction Tuning
  14. Supervised Fine-tuning (SFT).
  15. Aligning LLMs with human feedback.
  16. ...and 20 more sections

Figures (1)

  • Figure 1: Cross-lingual human feedback dataset. Given instructions and inputs written in English, both the accepted and rejected outputs are written in Chinese.