Parallel Scaling Law: Unveiling Reasoning Generalization through A Cross-Linguistic Perspective

TL;DR

This paper investigates whether reasoning abilities learned by English-centric RL-trained LRMs transfer across languages. It introduces the Multilingual Transferability Index (MTI) and conducts a three-stage program: Observational, Interventional, and Parallel Training studies. The findings reveal that cross-lingual transferability varies with model initialization, training paradigm, and language, and that English-centric models often overfit to language-specific patterns. A key contribution is the Parallel Training Study, which uncovers a First-Parallel Leap and a Parallel Scaling Law, showing a power-law relationship between the number of parallel languages and cross-lingual reasoning performance, alongside a Monolingual Generalization Gap. Collectively, the work highlights the limitations of current LRMs in language-agnostic reasoning and proposes practical strategies (e.g., Just Go Parallel) to improve multilingual reasoning generalization and inform future development of language-agnostic AI systems.

Abstract

Recent advancements in Reinforcement Post-Training (RPT) have significantly enhanced the capabilities of Large Reasoning Models (LRMs), sparking increased interest in the generalization of RL-based reasoning. While existing work has primarily focused on investigating its generalization across tasks or modalities, this study proposes a novel cross-linguistic perspective to investigate reasoning generalization. This raises a crucial question: $\textit{Does the reasoning capability achieved from English RPT effectively transfer to other languages?}$ We address this by systematically evaluating English-centric LRMs on multilingual reasoning benchmarks and introducing a metric to quantify cross-lingual transferability. Our findings reveal that cross-lingual transferability varies significantly across initial model, target language, and training paradigm. Through interventional studies, we find that models with stronger initial English capabilities tend to over-rely on English-specific patterns, leading to diminished cross-lingual generalization. To address this, we conduct a thorough parallel training study. Experimental results yield three key findings: $\textbf{First-Parallel Leap}$, a substantial leap in performance when transitioning from monolingual to just a single parallel language, and a predictable $\textbf{Parallel Scaling Law}$, revealing that cross-lingual reasoning transfer follows a power-law with the number of training parallel languages. Moreover, we identify the discrepancy between actual monolingual performance and the power-law prediction as $\textbf{Monolingual Generalization Gap}$, indicating that English-centric LRMs fail to fully generalize across languages. Our study challenges the assumption that LRM reasoning mirrors human cognition, providing critical insights for the development of more language-agnostic LRMs.

Figures (10)

• Figure 1: Cross-lingual reasoning transferability across open-source LRMs. The top subfigure shows the average Multilingual Transferability Index (MTI) of various English-centric LRMs across four benchmarks and eleven languages, with the x-axis representing the base models. The bottom subfigure presents the average Transferability Index (TI) performance of SFT- and RL-tuned models on individual languages on the MATH500 benchmark.
• Figure 2: The Impact of Different Initial Model Families on Interventional Study. Multilingual reasoning performance across languages on MATH500 benchmark, comparing the influence of model family using Qwen2.5-7B-Instruct and Llama3.1-8B-Instruct as initial models. "Base" represents the performance of the initial model, while "+GRPO" denotes performance after fine-tuning with GRPO on English data. The light red area denotes the improvement in accuracy between the "Base" and "+GRPO" models, while the light gray area represents the reduction in the off-target rate between the two.
• Figure 3: The Impact of Different Model Size on Interventional Study. Performance on various benchmarks across models of different sizes. "$\Delta \text{Performance}$" denotes the average difference in accuracy performance between the trained model and its initial model, averaged across both the training language and unseen languages, respectively.
• Figure 4: The Parallel Scaling Law in Multilingual Reasoning Performance. The x-axis Number of Training Languages is defined as English plus the specified number of parallel languages. "Experimental Data" shows the performance metrics of the model under different training numbers of parallel languages. The curves are fitted to the Experimental Data. "Monolingual Baseline" refers to fine-tuning on English data only, without parallel data. "First-Parallel Leap" denotes the performance difference between a model with one parallel language and the Monolingual Baseline.
• Figure 5: Accuracy difference comparison across parallel and unparallel data training.