Mitigating Overthinking in Large Reasoning Language Models via Reasoning Path Deviation Monitoring

Abstract

Large Reasoning Language Models (LRLMs) demonstrate impressive capabilities on complex tasks by utilizing long Chain-of-Thought reasoning. However, they are prone to overthinking, which generates redundant reasoning steps that degrade both performance and efficiency. Recently, early-exit strategies are proposed to mitigate overthinking by dynamically and adaptively terminating redundant reasoning. However, current early-exit methods either introduce extra training overhead by relying on proxy models or limit inference throughput due to the frequent content switching between reasoning and generating probing answers. Moreover, most early-exit methods harm LRLMs performance due to over-truncation. Our insight stems from an observation: overthinking often causes LRLMs to deviate from the correct reasoning path, which is frequently accompanied by high-entropy transition tokens. Given this, we propose an early-exit method deeply coupled with the native reasoning process, which leverages the path deviation index as a dedicated monitoring metric for the frequent occurrence of high-entropy transition tokens to dynamically detect and terminate overthinking trajectories. We conduct experiments across multiple benchmarks using LRLMs of different types and scales, and the results indicate that our method delivers the largest performance improvement over vanilla CoT compared to existing early-exit methods.

Figures (10)

• Figure 1: Reasoning trajectory dissection revealing the "overthinking" trap. The model initially achieves the correct result but fails due to a flawed verification loop. The frequent emergence of transition tokens (e.g., "Wait," "But") serves as a key indicator of reasoning deviation.
• Figure 2: Token Entropy Contribution Distribution. This figure illustrates the distribution of $45.9$ million tokens, sorted by their entropy values from low to high. The tokens are divided into 100 percentile bins, with the y-axis representing the percentage contribution of the total entropy within each bin relative to the aggregate entropy of all tokens.
• Figure 3: Visualization of High-Frequency Tokens among Top Entropy Contributors. This figure highlights the most frequent tokens within the $20\%$ of tokens that contributed most to the average entropy. Only tokens that constitute complete English words are retained.
• Figure 4: Overview of RPDI-EE. RPDI-EE performs continuous entropy scanning during CoT generation. For each new token, it measures local uncertainty via the Local Reasoning Density (LRD), defined as the average token entropy in a sliding window, and compares it to the global reasoning stability reflected by the Global Reasoning Baseline (GRB).
• Figure 5: Triggering Rates and Corrective Performance.