Scaling Reasoning, Losing Control: Evaluating Instruction Following in Large Reasoning Models

TL;DR

<3-5 sentence high-level summary> This work identifies a critical gap in evaluating instruction-following for math-focused large reasoning models and introduces MathIF, a benchmark built from Python-verifiable constraints to probe how well LRMs can follow user directives during math reasoning. Across 23 models and diverse problem sources, the authors show a robust intelligence–obedience trade-off: increasing reasoning strength and longer CoTs often degrade instruction adherence, sometimes with larger models not outperforming smaller, instruction-aware baselines. Through training analyses (SFT, RL) and inference-time interventions (CoT length control, constraint repetition), they demonstrate that reasoning improvements frequently come at the expense of obeying constraints, suggesting a need for new training paradigms that preserve instruction-following while maintaining reasoning capability. The findings have practical implications for alignment and safety in reasoning-centric LLMs and motivate development of instruction-aware reasoning methods.

Abstract

Instruction-following is essential for aligning large language models (LLMs) with user intent. While recent reasoning-oriented models exhibit impressive performance on complex mathematical problems, their ability to adhere to natural language instructions remains underexplored. In this work, we introduce MathIF, a dedicated benchmark for evaluating instruction-following in mathematical reasoning tasks. Our empirical analysis reveals a consistent tension between scaling up reasoning capacity and maintaining controllability, as models that reason more effectively often struggle to comply with user directives. We find that models tuned on distilled long chains-of-thought or trained with reasoning-oriented reinforcement learning often degrade in instruction adherence, especially when generation length increases. Furthermore, we show that even simple interventions can partially recover obedience, though at the cost of reasoning performance. These findings highlight a fundamental tension in current LLM training paradigms and motivate the need for more instruction-aware reasoning models. We release the code and data at https://github.com/TingchenFu/MathIF.

Figures (7)

• Figure 1: Performance of Instruct LLMs and LRMs on IFEval ifeval and FollowBench followbench.
• Figure 2: The accuracy on each subset MathIF averaged over 23 LRMs.
• Figure 3: The HAcc (solid line) and SAcc (dashed line) on the single/double/triple-constraint subset.
• Figure 4: Error set analysis for Qwen3-0.6B, DeepSeek-R1-Distill-Qwen-1.5B, Open-Reasoner-Zero-7B, and DeepSeek-R1-Distill-Llama-8B (from left to right).
• Figure 5: Relative correctness drop of four LRMs across five subsets.