Self-Improving Multilingual Long Reasoning via Translation-Reasoning Integrated Training

TL;DR

This work tackles the gap between reasoning and understanding in multilingual models by introducing TRIT, a self-improving reinforcement learning framework that jointly trains translation and multilingual reasoning without external data. TRIT operates in two stages: first strengthening cross-lingual reasoning with an accuracy-based filter, then translating English questions into the target language and training reasoning in that language, with a closed-loop feedback between translation quality and reasoning performance. Across MMATH and multiple backbone models, TRIT yields meaningful gains in multilingual reasoning quality and almost perfect language consistency, while also improving translation quality and cross-lingual alignment, with positive spillovers to general-domain text (FLORES-200). The results demonstrate that translation-trained representations and question-level alignment can robustly boost multilingual reasoning, offering a scalable path for robust cross-language mathematical and general-domain problem solving.

Abstract

Long reasoning models often struggle in multilingual settings: they tend to reason in English for non-English questions; when constrained to reasoning in the question language, accuracies drop substantially. The struggle is caused by the limited abilities for both multilingual question understanding and multilingual reasoning. To address both problems, we propose TRIT (Translation-Reasoning Integrated Training), a self-improving framework that integrates the training of translation into multilingual reasoning. Without external feedback or additional multilingual data, our method jointly enhances multilingual question understanding and response generation. On MMATH, our method outperforms multiple baselines by an average of 7 percentage points, improving both answer correctness and language consistency. Further analysis reveals that integrating translation training improves cross-lingual question alignment by over 10 percentage points and enhances translation quality for both mathematical questions and general-domain text, with gains up to 8.4 COMET points on FLORES-200.

Figures (11)

• Figure 1: The Framework of TRIT. Our framework consists of two stages: Cross-Lingual Reasoning filters questions by accuracy threshold $\theta$, and Translation-Reasoning Integration & Feedback trains both translation and target-language reasoning using filtered questions (Translation errors are denoted with red color, which results in wrong reasoning results, and get 0 as $r_\text{trans}$).
• Figure 2: Evolution of translation quality. (a) In-domain evaluation on MATH500 (Win/Tie/Lose rates vs. Base). (b) Cross-domain generalization on Flores200 (Comet Scores).
• Figure 3: Cross-lingual question alignment across model layers (DeepSeek-Distill-Qwen-1.5B). Layer-wise cosine similarity between English and target-language question representations for TRIT and External-Translation (ET, without translation training).
• Figure 4: Impact of Stage 1 Filtering Threshold ($\theta$) on Final Multilingual Reasoning Performance
• Figure 5: Translation quality correlates with reasoning accuracy. Distribution of translation quality (Win/Lose/Tie judged by DeepSeek-V3.2) for question pairs with (a) moderate accuracy differences ($\Delta\text{Acc} > 0.2$) and (b) critical failures (Acc = 0 vs. Acc > 0). Better translations consistently correspond to higher reasoning accuracy.