Cross-lingual Collapse: How Language-Centric Foundation Models Shape Reasoning in Large Language Models

TL;DR

Under RLVR, CoT in LLMs systematically drifts toward the pre-training dominant language as reasoning performance rises; English-centric priors, long-CoT GRPO optimization, task difficulty, and high-entropy decoding jointly amplify this drift, and the pattern persists beyond mathematics.

Abstract

Reinforcement learning with verifiable reward (RLVR) has been instrumental in eliciting strong reasoning capabilities from large language models (LLMs) via long chains of thought (CoT). During RLVR training, we formalize and systemically study an empirical phenomenon whereby a multilingual model's CoT reverts to its dominant pre-training language (e.g., English) even when prompted in another language, which we term Cross-lingual Collapse. Because the long-CoT regime magnifies exposure to linguistic priors, the underlying trade-off between maximizing reasoning depth and preserving target-language fidelity has remained under-characterized. To examine this trade-off, we train LLMs with Group-Relative Policy Optimization (GRPO) on translated versions of math datasets widely used to elicit long-CoT reasoning. Throughout training, we track both task accuracy and the language consistency of reasoning chains. Our experiments yield three findings: (i) under RLVR, CoT in LLMs systematically drifts toward the pre-training dominant language as reasoning performance rises; (ii) English-centric priors, long-CoT GRPO optimization, task difficulty, and high-entropy decoding jointly amplify this drift, and the pattern persists beyond mathematics; and (iii) interventions that favor target-language traces--via a language-consistency reward, decoding-time controls, or more balanced backbones--mitigate collapse but reveal a persistent performance-fidelity trade-off.

Figures (4)

• Figure 1: Illustration of Cross-lingual Collapse. We train Llama-$3.2$-$3$B Instruct with GRPO on a fully Ukrainian translation of GSM$8$K, seeking Ukrainian-only reasoning. (a) Chain-of-thought word-ratio in reward warding rollouts over training steps. In the grey band, the share of Ukrainian tokens plummets, while English abruptly dominates, signaling a language switch within the rollout reasoning trace. (b) Accuracy on the Ukrainian GSM$8$K. The sharp rise in accuracy aligns with the same $100$–$250$-step window, showing that the model scores higher once its reasoning drifts into English. (c) Representative responses at steps $100$ and $200$ (answer spans highlighted in purple). When the model reasons in Ukrainian it produces an incorrect answer, but after switching to English it solves the problem correctly, exemplifying the collapse from target-language reasoning to the pre-training-dominant language. The word ratio is measured during training from the rollout samples.
• Figure 2: Figures \ref{['fig:lancon_a']}–\ref{['fig:lancon_c']} compare Llama-$3.2$-$3$B Instruct trained with GRPO on the Ukrainian-translated GSM$8$K with and without the language-consistency reward (Lang loss). The language-consistency reward reliably preserves the target-language word ratio, yet it also dampens the accuracy gains that GRPO would otherwise deliver. In particular, Figures \ref{['fig:lancon_a']}–\ref{['fig:lancon_c']} show that the reward almost completely prevents cross-lingual collapse in the Ukrainian run—though at the cost of a modest drop in performance
• Figure 3: Rollout examples from GRPO training of Qwen-$2.5$$1.5$B on the Korean-translated GSM$8$K. Observe that the model often arrives at the right answer via English reasoning (non-target language); because any correct answer earns full reward, repeated reinforcement of such off-language traces gradually shifts the chain-of-thought word ratio away from Korean.
• Figure 4: We continued GRPO fine-tuning of the DeepSeek-R1-Distill Qwen model on the Korean-translated GSM$8$K dataset to encourage Korean chain-of-thought reasoning. As Figure \ref{['fig:distill_word_ratio']} shows, the distilled model still exhibits cross-lingual collapse during training.