EntroCut: Entropy-Guided Adaptive Truncation for Efficient Chain-of-Thought Reasoning in Small-scale Large Reasoning Models
Hongxi Yan, Qingjie Liu, Yunhong Wang
TL;DR
This paper tackles the inefficiency of long chain-of-thought reasoning in large reasoning models by using early-step output entropy as a confidence signal. It introduces EntroCut, a training-free method that adaptively truncates reasoning via a prefix-entropy probe, terminating when the probe's average entropy falls below a threshold $\tau$. A new efficiency metric, the Efficiency-Performance Ratio (EPR), quantifies token savings per unit accuracy loss. Empirical results across four math benchmarks and two model sizes show EntroCut achieves substantial token reductions with minimal accuracy sacrifice and superior EPR compared with baselines, demonstrating effective, practical inference-time efficiency gains for LRMs.
Abstract
Large Reasoning Models (LRMs) excel at complex reasoning tasks through extended chain-of-thought generation, but their reliance on lengthy intermediate steps incurs substantial computational cost. We find that the entropy of the model's output distribution in early reasoning steps reliably distinguishes correct from incorrect reasoning. Motivated by this observation, we propose EntroCut, a training-free method that dynamically truncates reasoning by identifying high-confidence states where reasoning can be safely terminated. To comprehensively evaluate the trade-off between efficiency and accuracy, we introduce the Efficiency-Performance Ratio (EPR), a unified metric that quantifies relative token savings per unit accuracy loss. Experiments on four benchmarks show that EntroCut reduces token usage by up to 40\% with minimal accuracy sacrifice, achieving superior efficiency-performance trade-offs compared with existing training-free methods. These results demonstrate that entropy-guided dynamic truncation provides a practical approach to mitigate the inefficiency of LRMs.