MathMixup: Boosting LLM Mathematical Reasoning with Difficulty-Controllable Data Synthesis and Curriculum Learning

TL;DR

MathMixup tackles the data bottleneck in LLM mathematical reasoning by introducing a difficulty-controllable data synthesis paradigm and the MathMixupQA dataset. Its three-stage pipeline—difficulty-controlled synthesis, quality-assured dataset construction with solution generation, and curriculum learning with graded data—enables explicit difficulty gradients and flexible blending with other datasets. Empirical results show consistent gains across seven benchmarks, achieving state-of-the-art performance when MathMixupQA is blended with MathFusionQA, and demonstrating the utility of structured, difficulty-aware training for LLM reasoning. The work advances data-centric curriculum learning and highlights the practical impact of controlled data generation on complex reasoning tasks.

Abstract

In mathematical reasoning tasks, the advancement of Large Language Models (LLMs) relies heavily on high-quality training data with clearly defined and well-graded difficulty levels. However, existing data synthesis methods often suffer from limited diversity and lack precise control over problem difficulty, making them insufficient for supporting efficient training paradigms such as curriculum learning. To address these challenges, we propose MathMixup, a novel data synthesis paradigm that systematically generates high-quality, difficulty-controllable mathematical reasoning problems through hybrid and decomposed strategies. Automated self-checking and manual screening are incorporated to ensure semantic clarity and a well-structured difficulty gradient in the synthesized data. Building on this, we construct the MathMixupQA dataset and design a curriculum learning strategy that leverages these graded problems, supporting flexible integration with other datasets. Experimental results show that MathMixup and its curriculum learning strategy significantly enhance the mathematical reasoning performance of LLMs. Fine-tuned Qwen2.5-7B achieves an average score of 52.6\% across seven mathematical benchmarks, surpassing previous state-of-the-art methods. These results fully validate the effectiveness and broad applicability of MathMixup in improving the mathematical reasoning abilities of LLMs and advancing data-centric curriculum learning.

Figures (5)

• Figure 1: Average accuracy comparison of different datasets and curriculum learning (CL) strategies on seven mathematical reasoning benchmarks across three LLMs (Qwen2.5-7B, InternLM2.5-7B, and LLaMA3.1-8B), with all datasets synthesized separately from MATH and AMC-AIME seeds. MathMixup consistently outperforms the MathFusion baseline, and curriculum learning (MathMixup-CL) further improves performance. Blending MathMixupQA and MathFusionQA (Blending) yields additional gains, while Blending-CL achieves the highest accuracy across all models and settings. These results demonstrate that difficulty-controllable data and curriculum learning are both effective individually, and their combination leads to the greatest improvements in LLM mathematical reasoning.
• Figure 2: Overview of the MathMixup pipeline. The process includes question pairs construction, question generation (hybrid and decomposed), question verification, solution generation with auxiliary information, and automated post-processing. MathMixup enables the synthesis of high-quality data with controllable difficulty, and supports curriculum learning strategies that further enhance LLM mathematical reasoning performance.
• Figure 3: Difficulty scores of different components in the MathMixupQA and MathFusionQA datasets. Note that Conditional, Parallel, and Sequential are synthesis methods proposed in MathFusion.
• Figure 4: Normalized performance on seven mathematical reasoning benchmarks for three LLMs using different data synthesis and training strategies. “Blending” denotes SFT on the mixed dataset of MathMixupQA and MathFusionQA, while “Blending-CL” denotes curriculum learning SFT on the same mixture.
• Figure 5: Effectiveness and efficiency of solution generation in MathMixup.