Multilingual Mathematical Reasoning: Advancing Open-Source LLMs in Hindi and English

TL;DR

The paper addresses the gap in multilingual mathematical reasoning for Hindi and English by proposing a unified training framework that combines a Decomposition Strategy, Structured Solution design, and Curriculum Learning. It develops IndiMathQA and enhances HAWP, then demonstrates that bilingual combined training improves cross-language reasoning while maintaining performance close to language-specific baselines. Empirical results show that WizardMath-7B, a lightweight open-source model, can outperform several baselines on English benchmarks and closely match Hindi performance when using the proposed methods, with gains up to $+6\%$ over strong baselines. Overall, the work demonstrates that principled decomposition, structured reasoning prompts, and curriculum-based multilingual fine-tuning can significantly boost mathematical problem solving in resource-efficient open-source models, enabling practical multilingual math support at scale.

Abstract

Large Language Models (LLMs) excel in linguistic tasks but struggle with mathematical reasoning, particularly in non English languages like Hindi. This research aims to enhance the mathematical reasoning skills of smaller, resource efficient open-source LLMs in both Hindi and English. We evaluate models like OpenHathi 7B, LLaMA-2 7B, WizardMath 7B, Mistral 7B, LLeMMa 7B, MAmmoTH 7B, Gemini Pro, and GPT-4 using zero-shot, few-shot chain-of-thought (CoT) methods, and supervised fine-tuning. Our approach incorporates curriculum learning, progressively training models on increasingly difficult problems, a novel Decomposition Strategy to simplify complex arithmetic operations, and a Structured Solution Design that divides solutions into phases. Our experiments result in notable performance enhancements. WizardMath 7B exceeds Gemini's accuracy on English datasets by +6% and matches Gemini's performance on Hindi datasets. Adopting a bilingual approach that combines English and Hindi samples achieves results comparable to individual language models, demonstrating the capability to learn mathematical reasoning in both languages. This research highlights the potential for improving mathematical reasoning in open-source LLMs.

Figures (6)

• Figure 1: Curriculum Learning with Structured Solutions: A Comprehensive Framework to Gradually Guide Models Through Complex Mathematical Challenges.
• Figure 2: Overall Methodology: The top section illustrates our primary approach, which combines Curriculum Learning and Bilingual Integrated Training. The bottom section depicts the process of applying the decomposition strategy to the HAWP dataset.
• Figure 3: Bar graph showing topic distribution across each difficulty level extracted from the MATH dataset
• Figure 4: Bar graph showing topic distribution across each difficulty level extracted from the IndiMathQA dataset
• Figure 5: Heatmap showing English mathematical performance across Difficulties (Easy, Medium, Hard) on each Mathematical Models.