MathFimer: Enhancing Mathematical Reasoning by Expanding Reasoning Steps through Fill-in-the-Middle Task

TL;DR

This paper introduces MathFimer, a novel framework for mathematical reasoning step expansion inspired by the''Fill-in-the-middle''task from code reasoning", and offers a practical, scalable solution for enhancing mathematical reasoning capabilities in LLMs without relying on powerful external models or expensive inference procedures.

Abstract

Mathematical reasoning represents a critical frontier in advancing large language models (LLMs). While step-by-step approaches have emerged as the dominant paradigm for mathematical problem-solving in LLMs, the quality of reasoning steps in training data fundamentally constrains the performance of the models. Recent studies have demonstrated that more detailed intermediate steps can enhance model performance, yet existing methods for step expansion either require more powerful external models or incur substantial computational costs. In this paper, we introduce MathFimer, a novel framework for mathematical reasoning step expansion inspired by the ''Fill-in-the-middle'' task from code reasoning. By decomposing solution chains into prefix-suffix pairs and training models to reconstruct missing intermediate steps, we develop a specialized model, MathFimer-7B, on our carefully curated NuminaMath-FIM dataset. We then apply these models to enhance existing mathematical reasoning datasets by inserting detailed intermediate steps into their solution chains, creating MathFimer-expanded versions. Through comprehensive experiments on multiple mathematical reasoning datasets, including MathInstruct, MetaMathQA and etc., we demonstrate that models trained on MathFimer-expanded data consistently outperform their counterparts trained on original data across various benchmarks such as GSM8K and MATH. Our approach offers a practical, scalable solution for enhancing mathematical reasoning capabilities in LLMs without relying on powerful external models or expensive inference procedures.

Figures (5)

• Figure 1: We developed MathFimer inspired by the fill-in-the-middle task in code reasoning of LLMs. Panel \ref{['fig:fim_code']} demonstrates an example where the FIM model completes a given code context, while Panel \ref{['fig:fim_math']} shows how MathFimer, as proposed in this paper, extends the steps of an existing step-by-step answer.
• Figure 2: An overview of our work. The left part illustrates how we construct FIM training data from existing CoT data and train FIM models, MathFimer, which works on chain-of-thought. The right part demonstrates the process where MathFimer is used to expand the steps of existing CoT data for more detailed reasoning.
• Figure 3: An example of NuminaMath-FIM. The left side represents a mathematical problem and its corresponding solution from NuminaMath-CoT, while the right side shows the FIM data constructed from it. The underlined portion represents a randomly selected step from all the steps, with the blue tokens <|fim_prefix|>, <|fim_suffix|>, and <|fim_middle|> being three special tokens. During supervised fine-tuning, we only compute the loss for the underlined portion.
• Figure 4: PRM score distribution before and after step insertion with MathFimer-7B. The PRM scores range from 0 to 1.
• Figure 5: Prompt for zero-shot prompt-based step fill.