Improving Multilingual Instruction Finetuning via Linguistically Natural and Diverse Datasets

TL;DR

This work tackles the English-dominated landscape of instruction fine-tuning by introducing a five-stage framework that generates linguistically natural, diverse multilingual IFT data using monolingual corpora and English-focused LLMs, with a scoring-based quality filter. By translating non-English responses to English, producing English instructions, scoring for quality and diversity, and translating back, the method creates about 500k high-quality IRT examples per language from 1M fragments. Empirically, models trained on the generated IFT data outperform translation- and templated baselines on both generative tasks (XLSUM, FLORES-200) and a discriminative task (MMLU), often with smaller data footprints, underscoring the value of language-native instruction data. The approach offers a practical, scalable solution to expanding multilingual instruction-following capabilities and sets a foundation for broader language coverage in LLMs.

Abstract

Advancements in Large Language Models (LLMs) have significantly enhanced instruction-following capabilities. However, most Instruction Fine-Tuning (IFT) datasets are predominantly in English, limiting model performance in other languages. Traditional methods for creating multilingual IFT datasets such as translating existing English IFT datasets or converting existing NLP datasets into IFT datasets by templating, struggle to capture linguistic nuances and ensure prompt (instruction) diversity. To address this issue, we propose a novel method for collecting multilingual IFT datasets that preserves linguistic naturalness and ensures prompt diversity. This approach leverages English-focused LLMs, monolingual corpora, and a scoring function to create high-quality, diversified IFT datasets in multiple languages. Experiments demonstrate that LLMs finetuned using these IFT datasets show notable improvements in both generative and discriminative tasks, indicating enhanced language comprehension by LLMs in non-English contexts. Specifically, on the multilingual summarization task, LLMs using our IFT dataset achieved 17.57% and 15.23% improvements over LLMs fine-tuned with translation-based and template-based datasets, respectively.

Figures (8)

• Figure 1: The incorrectly translated Telugu instruction-response pair is from the Aya collection 2024aya, which was translated from an English instruction-response pair in the Dolly v2 dataset DatabricksBlog2023DollyV2. The correct Telugu instruction-response pair was provided by a native Telugu speaker.
• Figure 2: Lack of diversity in templated datasets: The template created by human annotators has been repeated several thousand times in the templated adversarial QA dataset from the Aya collection 2024aya
• Figure 3: Overview of the proposed method: (A) Select Response, (B) Translating Response to English, (C) Generating English instructions using the English Response and task-specific prompt, (D) Scoring the generated English instruction against the translated response, and (E) Translating the English instruction back to the language of the original response.
• Figure 4: Instruction diversity in the generated IFT dataset. The inner circle displays common root verbs, while the outer circle shows the corresponding noun objects, based on approximately 15 percent of instructions generated across 4 languages. The figure only represents 13.1% of verb-noun pairs since not all instructions have the parsed verb-noun structure.
• Figure 5: Scores assigned by LLM judge on Instruction-Response pairs. The scores are averaged across all languages.