ENTRA: Entropy-Based Redundancy Avoidance in Large Language Model Reasoning

TL;DR

This work addresses overthinking in Large Reasoning Models (LRMs), where extended Chain-of-Thought reasoning leads to unnecessary token generation without commensurate gains. It proposes ENTRA, a unified training framework that combines Bidirectional Importance Estimation (BIE) with an entropy-based redundancy reward, optimized via GRPO reinforcement learning to prune redundant reasoning while preserving accuracy. On math reasoning benchmarks, ENTRA reduces reasoning length by 37% to 53% and maintains or improves accuracy, demonstrating a principled approach to redundancy-aware reasoning. The method offers a generalizable path toward more efficient and interpretable reasoning in LRMs, with potential applicability beyond mathematical tasks.

Abstract

Large Reasoning Models (LRMs) often suffer from overthinking, generating unnecessarily long reasoning chains even for simple tasks. This leads to substantial computational overhead with limited performance gain, primarily due to redundant verification and repetitive generation. While prior work typically constrains output length or optimizes correctness, such coarse supervision fails to guide models toward concise yet accurate inference. In this paper, we propose ENTRA, an entropy-based training framework that suppresses redundant reasoning while preserving performance. ENTRA first estimates the token-level importance using a lightweight Bidirectional Importance Estimation (BIE) method, which accounts for both prediction confidence and forward influence. It then computes a redundancy reward based on the entropy of low-importance tokens, normalized by its theoretical upper bound, and optimizes this reward via reinforcement learning. Experiments on mathematical reasoning benchmarks demonstrate that ENTRA reduces output length by 37% to 53% with no loss-and in some cases, gains-in accuracy. Our approach offers a principled and efficient solution to reduce overthinking in LRMs, and provides a generalizable path toward redundancy-aware reasoning optimization.

Figures (6)

• Figure 1: After training on the basis of Qwen3-8b, our method can effectively reduce the steps of excessive inference and guarantee the correct answer.
• Figure 2: Illustration of our Bidirectional Importance Estimation (BIE). The importance of token $x_3$ is computed by combining its self-information $\log P(x_3 \mid x_{\leq 3})$ (left) with its influence on future tokens, estimated via average attention weights $\mu_3$ (right).
• Figure 3: Compression analysis of each method based on Qwen3-8b
• Figure 4: Compression analysis of each method based on Qwen3-4b
• Figure 5: Case study comparing the reasoning trajectories of ENTRA-8B and LC-R1-8B.