Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Stop Unnecessary Reflection: Training LRMs for Efficient Reasoning with Adaptive Reflection and Length Coordinated Penalty

Zewei Yu, Lirong Gao, Yuke Zhu, Bo Zheng, Sheng Guo, Haobo Wang, Junbo Zhao

TL;DR

Experimental results show that ARLCP achieves a superior efficiency-accuracy trade-off compared to existing approaches, and a key innovations: a reflection penalty that adaptively curtails unnecessary reflective steps while preserving essential reasoning.

Abstract

Large Reasoning Models (LRMs) have demonstrated remarkable performance on complex reasoning tasks by employing test-time scaling. However, they often generate over-long chains-of-thought that, driven by substantial reflections such as repetitive self-questioning and circular reasoning, lead to high token consumption, substantial computational overhead, and increased latency without improving accuracy, particularly in smaller models. Our observation reveals that increasing problem complexity induces more excessive and unnecessary reflection, which in turn reduces accuracy and increases token overhead. To address this challenge, we propose Adaptive Reflection and Length Coordinated Penalty (ARLCP), a novel reinforcement learning framework designed to dynamically balance reasoning efficiency and solution accuracy. ARLCP introduces two key innovations: (1) a reflection penalty that adaptively curtails unnecessary reflective steps while preserving essential reasoning, and (2) a length penalty calibrated to the estimated complexity of the problem. By coordinating these penalties, ARLCP encourages the model to generate more concise and effective reasoning paths. We evaluate our method on five mathematical reasoning benchmarks using DeepSeek-R1-Distill-Qwen-1.5B and DeepSeek-R1-Distill-Qwen-7B models. Experimental results show that ARLCP achieves a superior efficiency-accuracy trade-off compared to existing approaches. For the 1.5B model, it reduces the average response length by 53.1% while simultaneously improving accuracy by 5.8%. For the 7B model, it achieves a 35.0% reduction in length with a 2.7% accuracy gain. The code is released at https://github.com/ZeweiYu1/ARLCP .

Stop Unnecessary Reflection: Training LRMs for Efficient Reasoning with Adaptive Reflection and Length Coordinated Penalty

TL;DR

Experimental results show that ARLCP achieves a superior efficiency-accuracy trade-off compared to existing approaches, and a key innovations: a reflection penalty that adaptively curtails unnecessary reflective steps while preserving essential reasoning.

Abstract

Large Reasoning Models (LRMs) have demonstrated remarkable performance on complex reasoning tasks by employing test-time scaling. However, they often generate over-long chains-of-thought that, driven by substantial reflections such as repetitive self-questioning and circular reasoning, lead to high token consumption, substantial computational overhead, and increased latency without improving accuracy, particularly in smaller models. Our observation reveals that increasing problem complexity induces more excessive and unnecessary reflection, which in turn reduces accuracy and increases token overhead. To address this challenge, we propose Adaptive Reflection and Length Coordinated Penalty (ARLCP), a novel reinforcement learning framework designed to dynamically balance reasoning efficiency and solution accuracy. ARLCP introduces two key innovations: (1) a reflection penalty that adaptively curtails unnecessary reflective steps while preserving essential reasoning, and (2) a length penalty calibrated to the estimated complexity of the problem. By coordinating these penalties, ARLCP encourages the model to generate more concise and effective reasoning paths. We evaluate our method on five mathematical reasoning benchmarks using DeepSeek-R1-Distill-Qwen-1.5B and DeepSeek-R1-Distill-Qwen-7B models. Experimental results show that ARLCP achieves a superior efficiency-accuracy trade-off compared to existing approaches. For the 1.5B model, it reduces the average response length by 53.1% while simultaneously improving accuracy by 5.8%. For the 7B model, it achieves a 35.0% reduction in length with a 2.7% accuracy gain. The code is released at https://github.com/ZeweiYu1/ARLCP .
Paper Structure (41 sections, 14 equations, 7 figures, 6 tables)

This paper contains 41 sections, 14 equations, 7 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Realated Work
  3. Large Reasoning Models.
  4. Efficient Reasoning for LRMs.
  5. Motivation and Observation
  6. Reflection correlates with problem complexity.
  7. Over-reflection leads to incorrect responses and inefficiency.
  8. Key insights and motivation.
  9. Methodology
  10. Preliminary
  11. Adaptive Reflection and Length Coordinated Penalty (ARLCP)
  12. The complexity of each problem $\lambda$.
  13. Reflection penalty.
  14. Length penalty.
  15. Experiments
  16. ...and 26 more sections

Figures (7)

  • Figure 1: Average reflection token counts statistics.
  • Figure 2: Output statistics of two models on the AIME 2024-2025 datasets. The reported metrics include average response tokens (left), average reflection token count (medium) for both correct and incorrect answers, and accuracy trend with different reflection token count intervals.
  • Figure 3: The framework of ARLCP. It adaptively imposes a reflection penalty according to the complexity of each problem, supplemented by a length penalty, allowing the LRM to flexibly reduce unnecessary reflection and minimize token consumption.
  • Figure 4: The analysis of accuracy, length, and reflection for model responses on five benchmarks (AMC 2023, AIME 2024, AIME 2025, GSM8K, and MATH 500) across different training steps.
  • Figure 5: Comparison of average reflection token count between vanilla models and ARLCP.
  • ...and 2 more figures