Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multilingual Arbitrage: Optimizing Data Pools to Accelerate Multilingual Progress

Ayomide Odumakinde, Daniel D'souza, Pat Verga, Beyza Ermis, Sara Hooker

TL;DR

The paper tackles the challenge of multilingual model training with a single teacher by introducing multilingual arbitrage, a strategy that routes synthetic data generation across a pool of diverse models to exploit language-specific strengths. The method reframes distillation as an optimization over which teacher to use for each prompt, employing routing methods (fixed, reward-based, and learned) and a teacher pool that includes multilingual, geo-cluster, and monolingual models. Experiments across 15 languages and 9 models show that arbitrage substantially surpasses single-teacher baselines, with reward-based routing achieving up to a 56.5% average win-rate improvement over the best single teacher and mid-resource languages benefiting the most. Beyond open-ended generation, the approach also improves discriminative tasks and yields richer textual characteristics (more tokens, higher lexical diversity, and readable yet more complex text), illustrating the practical value of strategic sampling in multilingual synthetic data generation. The work highlights the potential of LLM ensembles to accelerate multilingual progress and suggests directions for scaling and safety considerations in future work.

Abstract

The use of synthetic data has played a critical role in recent state-of-art breakthroughs. However, overly relying on a single oracle teacher model to generate data has been shown to lead to model collapse and invite propagation of biases. These limitations are particularly evident in multilingual settings, where the absence of a universally effective teacher model that excels across all languages presents significant challenges. In this work, we address these extreme difference by introducing "multilingual arbitrage", which capitalizes on performance variations between multiple models for a given language. To do so, we strategically route samples through a diverse pool of models, each with unique strengths in different languages. Across exhaustive experiments on state-of-art models, our work suggests that arbitrage techniques allow for spectacular gains in performance that far outperform relying on a single teacher. In particular, compared to the best single teacher, we observe gains of up to 56.5% improvement in win rates averaged across all languages when switching to multilingual arbitrage. We observe the most significant gains for the least resourced languages in our pool.

Multilingual Arbitrage: Optimizing Data Pools to Accelerate Multilingual Progress

TL;DR

The paper tackles the challenge of multilingual model training with a single teacher by introducing multilingual arbitrage, a strategy that routes synthetic data generation across a pool of diverse models to exploit language-specific strengths. The method reframes distillation as an optimization over which teacher to use for each prompt, employing routing methods (fixed, reward-based, and learned) and a teacher pool that includes multilingual, geo-cluster, and monolingual models. Experiments across 15 languages and 9 models show that arbitrage substantially surpasses single-teacher baselines, with reward-based routing achieving up to a 56.5% average win-rate improvement over the best single teacher and mid-resource languages benefiting the most. Beyond open-ended generation, the approach also improves discriminative tasks and yields richer textual characteristics (more tokens, higher lexical diversity, and readable yet more complex text), illustrating the practical value of strategic sampling in multilingual synthetic data generation. The work highlights the potential of LLM ensembles to accelerate multilingual progress and suggests directions for scaling and safety considerations in future work.

Abstract

The use of synthetic data has played a critical role in recent state-of-art breakthroughs. However, overly relying on a single oracle teacher model to generate data has been shown to lead to model collapse and invite propagation of biases. These limitations are particularly evident in multilingual settings, where the absence of a universally effective teacher model that excels across all languages presents significant challenges. In this work, we address these extreme difference by introducing "multilingual arbitrage", which capitalizes on performance variations between multiple models for a given language. To do so, we strategically route samples through a diverse pool of models, each with unique strengths in different languages. Across exhaustive experiments on state-of-art models, our work suggests that arbitrage techniques allow for spectacular gains in performance that far outperform relying on a single teacher. In particular, compared to the best single teacher, we observe gains of up to 56.5% improvement in win rates averaged across all languages when switching to multilingual arbitrage. We observe the most significant gains for the least resourced languages in our pool.
Paper Structure (24 sections, 3 equations, 11 figures, 11 tables)

This paper contains 24 sections, 3 equations, 11 figures, 11 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. Routing Methods
  4. Experimental Setup
  5. Baselines
  6. Routing Teacher Pools
  7. Student Model
  8. Evaluations
  9. Results and Discussion
  10. Multilingual Arbitrage Performance
  11. Language and Routing Analysis
  12. Textual Characteristics
  13. Related Work
  14. Conclusion
  15. Limitations
  16. ...and 9 more sections

Figures (11)

  • Figure 1: Overview of Multilingual Arbitrage. Instead of relying on a single "oracle" teacher, multilingual arbitrage re-frames the distillation problem as learning how to optimize sampling for a desired part of the data distribution from an ensemble of teachers.
  • Figure 2: Win rate % comparison of Fixed Routing, Reward-Based Routing and Learned Routing against multiple Single Teacher Models. The x-axis represents the single teacher used to generate the synthetic data for training. We observe that all multilingual arbitrage strategies outperform all single teachers with the largest gains observed for reward-based routing. All values are percentages and aggregated across 7 languages: Arabic, Chinese, English, French, German, Turkish, and Ukrainian.
  • Figure 3: Win rates % of students trained with different routing strategies: Comparison of router-trained students to those trained with random routing. The largest gains are observed for reward-based routing with a win-loss diff of 30.6%. All values are percentages and aggregated over 7 languages.
  • Figure 4: Win rate Changes Across Language Resource Level: Comparison of the Mid-Resource Languages and High-Resource Languages win rates against Single Teachers (results are the average of Aya 23, Llama 3 and Gemma 2 single teachers). Mid-resource languages consist of Turkish and Ukrainian and high-resource languages are English, German, French, Chinese and Arabic.
  • Figure 5: Model Composition per Language: Here we analyze the model routing distribution of a dataset constructed with Reward-Based Routing. The values represent the percentage of prompts routed to a given model for the particular language.
  • ...and 6 more figures