Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The Reasoning Lingua Franca: A Double-Edged Sword for Multilingual AI

Alan Saji, Raj Dabre, Anoop Kunchukuttan, Ratish Puduppully

TL;DR

This work investigates whether LRMs should reason in English or in the question's language for multilingual tasks, using MGSM and GPQA Diamond to compare final answers and the cognitive features of reasoning traces. It employs language-forcing prefixes and system prompts to induce language-specific reasoning, with four repeated trials across open-weight models. Findings show English reasoning often yields higher accuracy and richer cognitive behaviors, but is vulnerable to Lost in Translation, especially for lower-resource languages and harder tasks. The study introduces LiT as a metric and demonstrates a need for native-language reasoning benchmarks, data, and training objectives to preserve language-specific nuance and reduce translation-induced errors. Overall, the results argue for developing robust native-language reasoning capabilities alongside English-centered approaches to improve multilingual AI reliability.

Abstract

Large Reasoning Models (LRMs) achieve strong performance on mathematical, scientific, and other question-answering tasks, but their multilingual reasoning abilities remain underexplored. When presented with non-English questions, LRMs often default to reasoning in English, raising concerns about interpretability and the handling of linguistic and cultural nuances. We systematically compare an LRM's reasoning in English versus the language of the question. Our evaluation spans two tasks: MGSM and GPQA Diamond. Beyond measuring answer accuracy, we also analyze cognitive attributes in the reasoning traces. We find that English reasoning traces exhibit a substantially higher presence of these cognitive behaviors, and that reasoning in English generally yields higher final-answer accuracy, with the performance gap increasing as tasks become more complex. However, this English-centric strategy is susceptible to a key failure mode - getting "Lost in Translation," where translation steps lead to errors that would have been avoided by question's language reasoning.

The Reasoning Lingua Franca: A Double-Edged Sword for Multilingual AI

TL;DR

This work investigates whether LRMs should reason in English or in the question's language for multilingual tasks, using MGSM and GPQA Diamond to compare final answers and the cognitive features of reasoning traces. It employs language-forcing prefixes and system prompts to induce language-specific reasoning, with four repeated trials across open-weight models. Findings show English reasoning often yields higher accuracy and richer cognitive behaviors, but is vulnerable to Lost in Translation, especially for lower-resource languages and harder tasks. The study introduces LiT as a metric and demonstrates a need for native-language reasoning benchmarks, data, and training objectives to preserve language-specific nuance and reduce translation-induced errors. Overall, the results argue for developing robust native-language reasoning capabilities alongside English-centered approaches to improve multilingual AI reliability.

Abstract

Large Reasoning Models (LRMs) achieve strong performance on mathematical, scientific, and other question-answering tasks, but their multilingual reasoning abilities remain underexplored. When presented with non-English questions, LRMs often default to reasoning in English, raising concerns about interpretability and the handling of linguistic and cultural nuances. We systematically compare an LRM's reasoning in English versus the language of the question. Our evaluation spans two tasks: MGSM and GPQA Diamond. Beyond measuring answer accuracy, we also analyze cognitive attributes in the reasoning traces. We find that English reasoning traces exhibit a substantially higher presence of these cognitive behaviors, and that reasoning in English generally yields higher final-answer accuracy, with the performance gap increasing as tasks become more complex. However, this English-centric strategy is susceptible to a key failure mode - getting "Lost in Translation," where translation steps lead to errors that would have been avoided by question's language reasoning.
Paper Structure (27 sections, 13 figures, 5 tables)

This paper contains 27 sections, 13 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Experimental Settings
  5. Results
  6. Analyzing cognitive behaviors in the reasoning trace
  7. Making a case for reasoning in the Question's language
  8. Conclusion
  9. Limitations
  10. Ethics Statement
  11. Acknowledgments
  12. Related Work - additional details
  13. Experimental settings: Additional details
  14. Dataset
  15. Models
  16. ...and 12 more sections

Figures (13)

  • Figure 1: MGSM - Reasoning in the Question’s Language. Final answer accuracy of LRMs on MGSM when reasoning in question's language. Y-axis: models; X-axis: languages; each cell reports the avg@4 accuracy. Standard deviations are below 0.05 (c.f. Appendix \ref{['sec: native_cot_reasoning_std']}).
  • Figure 2: Reasoning in English vs Question's Language is compared via final answer accuracy for Qwen3 32B for MGSM and GPQA diamond task. Error bars denote standard deviation.
  • Figure 3: GPQA diamond - Reasoning in the Question’s Language. Final answer accuracy of LRMs on GPQA diamond when reasoning in question's language. Y-axis: models; X-axis: languages; each cell reports the avg@4 accuracy and standard deviation.
  • Figure 4: Cognitive behaviors in the reasoning chain are averaged and compared for each language across reasoning in English and in the question's language. Error bars denote standard deviation. Model: Qwen QwQ-32B
  • Figure 5: Lost in Translation. Visualization of fraction of incorrect answers occurring due to errors in translation when reasoning in English. Error bars represents standard deviation.
  • ...and 8 more figures