Efficient Reasoning for Large Reasoning Language Models via Certainty-Guided Reflection Suppression

TL;DR

The paper addresses the overthinking tendency of Large Reasoning Language Models (LRLMs) that arises from reflection-driven reasoning steps. It introduces Certainty-Guided Reflection Suppression (CGRS), a training-free, model-agnostic decoding framework that uses checkpoint-based certainty estimation and dynamic suppression of reflection triggers to curb redundant thinking. Through experiments on four benchmarks (AIME24, AMC23, MATH500, GPQA-D) and across diverse open-source models (4B–32B), CGRS achieves substantial token reductions (18.5%–41.9%) with minimal accuracy loss and outperforms existing efficient-reasoning baselines. The approach offers a practical, scalable path to more cost-effective LRLM deployment by controlling reflectivity during inference without retraining or architectural changes.

Abstract

Recent Large Reasoning Language Models (LRLMs) employ long chain-of-thought reasoning with complex reflection behaviors, typically signaled by specific trigger words (e.g., "Wait" and "Alternatively") to enhance performance. However, these reflection behaviors can lead to the overthinking problem where the generation of redundant reasoning steps that unnecessarily increase token usage, raise inference costs, and reduce practical utility. In this paper, we propose Certainty-Guided Reflection Suppression (CGRS), a novel method that mitigates overthinking in LRLMs while maintaining reasoning accuracy. CGRS operates by dynamically suppressing the model's generation of reflection triggers when it exhibits high confidence in its current response, thereby preventing redundant reflection cycles without compromising output quality. Our approach is model-agnostic, requires no retraining or architectural modifications, and can be integrated seamlessly with existing autoregressive generation pipelines. Extensive experiments across four reasoning benchmarks (i.e., AIME24, AMC23, MATH500, and GPQA-D) demonstrate CGRS's effectiveness: it reduces token usage by an average of 18.5% to 41.9% while preserving accuracy. It also achieves the optimal balance between length reduction and performance compared to state-of-the-art baselines. These results hold consistently across model architectures (e.g., DeepSeek-R1-Distill series, QwQ-32B, and Qwen3 family) and scales (4B to 32B parameters), highlighting CGRS's practical value for efficient reasoning.

Figures (4)

• Figure 1: Accuracy and output length of the Vanilla and proposed CGRS methods across different models, averaged over three mathematical reasoning benchmarks (AIME24, AMC23, and MATH500) and one scientific reasoning benchmark (GPQA-D). CGRS achieves an average token reduction of $18.5-41.9\%$ while maintaining performance.
• Figure 2: Comparison of Vanilla and CGRS methods in terms of the frequency of reflection triggers (left) and the answer length distribution (right).
• Figure 3: Comparison of the responses from Vanilla and the proposed CGRS (with $\delta$ = 0.9) methods on an example from the MATH500 dataset using the DeepSeek-R1-Distill-Qwen-7B model.
• Figure 4: Certainty distributions of probes with correct and incorrect results on AIME24, AMC23, and MATH500.