Why Do Multilingual Reasoning Gaps Emerge in Reasoning Language Models?

TL;DR

This work identifies understanding failures as the primary cause of multilingual reasoning gaps in reasoning language models, showing that inputs poorly mapped into the model’s dominant English reasoning space lead to weaker cross-language performance. It formalizes multilingual reasoning as a three-stage process (understanding, reasoning in a dominant language, generation) and uses a residual-based analysis to attribute the gap, finding understanding failures dominate across models and tasks. The authors demonstrate that understanding failures can be detected with supervised detectors, LLM-based signals, or token-probability cues, with supervised detectors offering the best accuracy and cross-language generalization. They then propose Selective Translation, which translates multilingual inputs to English only when a misunderstanding is detected, achieving near full-translation performance while translating roughly 20% of inputs. The results highlight a practical pathway to more equitable multilingual reasoning, showing that detection and selective mitigation can substantially bridge the multilingual gap and suggesting future work to integrate these signals into model training.

Abstract

Reasoning language models (RLMs) achieve strong performance on complex reasoning tasks, yet they still suffer from a multilingual reasoning gap, performing better in high-resource languages than in low-resource ones. While recent efforts have reduced this gap, its underlying causes remain largely unexplored. In this paper, we address this by showing that the multilingual reasoning gap largely stems from failures in language understanding-the model's inability to represent the multilingual input meaning into the dominant language (i.e., English) within its reasoning trace. This motivates us to examine whether understanding failures can be detected, as this ability could help mitigate the multilingual reasoning gap. To this end, we evaluate a range of detection methods and find that understanding failures can indeed be identified, with supervised approaches performing best. Building on this, we propose Selective Translation, a simple yet effective strategy that translates the multilingual input into English only when an understanding failure is detected. Experimental results show that Selective Translation bridges the multilingual reasoning gap, achieving near full-translation performance while using translation for only about 20% of inputs. Together, our work demonstrates that understanding failures are the primary cause of the multilingual reasoning gap and can be detected and selectively mitigated, providing key insight into its origin and a promising path toward more equitable multilingual reasoning. Our code and data are publicly available at https://github.com/deokhk/RLM_analysis.

Figures (16)

• Figure 1: Understanding failure in Qwen3-4B: the model shows confusion when interpreting the Swahili input (e.g., “This is confusing…”) and ignores the “1 bad orange” condition, leading to an incorrect answer.
• Figure 2: Weighted residual-based shares of Understanding, Reasoning, and Generation in the input language to the overall multilingual reasoning gap. Across models and datasets, failures in Understanding generally dominate the gap.
• Figure 3: Scatter plot of Reasoning Performance Ratio on Polymath-Low vs. Translation quality on FLORES-200 (measured by GEMBA-DA in the xx$\rightarrow$en direction). A global linear trend line shows a strong positive Pearson correlation ($r=0.951$)
• Figure 4: F1 scores for understanding failure detection on unseen languages (fr, mr, wo) across two benchmarks on Qwen3-4B.
• Figure 5: F1 score of understanding detection on Polymath-Low with Qwen3-4B, measured with varying the maximum reasoning trace token length.