SmartSwitch: Advancing LLM Reasoning by Overcoming Underthinking via Promoting Deeper Thought Exploration

TL;DR

This work tackles underthinking in LongCoT reasoning by introducing SmartSwitch, a plug-and-play inference framework that continuously monitors an LLM's reasoning and intervenes to deepen promising but prematurely abandoned thoughts. It comprises a perception module that detects thought switches and scores the preceding thought with a Process Reward Model (PRM), and an intervention module that backtracks and injects a deepen prompt when a high-potential path is identified, all without retraining. The framework uses an adaptive process-division strategy and a long-context PRM (Universal-PRM-7B) with a promising-score threshold of $0.7$, demonstrating substantial accuracy gains on five math benchmarks across models from $1.5$B to $32$B, while also reducing token usage and latency. Empirical results show that SmartSwitch not only improves challenging problem solving but also preserves correct solutions, suggesting broad applicability to complex reasoning tasks. Overall, SmartSwitch offers a practical, model-agnostic method to enhance the depth and reliability of LLM reasoning with potential impact across domains requiring robust multi-step problem solving.

Abstract

The long chain-of-thought (LongCoT) capability is central to the recent breakthroughs achieved by large language models in complex reasoning tasks. However, the accompanying issue of ''underthinking'', where models exhibit shallow reasoning by frequently switching thoughts without sufficient exploration, limits both performance and token efficiency. To address this problem, we propose a simple yet effective reasoning strategy: the SmartSwitch inference framework. This framework can be easily integrated into any large language model as a plug-and-play solution, continuously monitoring the model's reasoning process to detect underthinking and guide it toward deeper exploration of promising but overlooked thoughts. Specifically, the perception module identifies points where thoughts switch and evaluates the potential of the preceding thought using an off-the-shelf process reward model (PRM). If a high-potential thought is found to be prematurely abandoned, the intervention module interrupts the ongoing inference, backtracks to the point before the switch, and inserts a "deepening prompt" to encourage further exploration along that promising path. Extensive experiments on challenging mathematical reasoning benchmarks demonstrate that our method significantly enhances the performance of various large language models of different sizes.

Figures (8)

• Figure 1: Qualitative and Quantitative illustration for the "underthinking problem". (a) presents an example with the underthinking phenomenon sampled from DeepSeek-R1 guo2025deepseekr1. The full response consists of 74 different thoughts, each with a relatively short length (around 150 tokens). (b) shows the "Underthinking Frequency" metric $\text{UF}(L)$ (defined in Eq.\ref{['eq:uf']}) of six mainstream LongCoT LLMs at different values of length threshold $L$. The results show that underthinking is widespread in all models.
• Figure 2: (a) Underthinking frequency increases with problem difficulty on the MATH-500 dataset hendrycks2021math. (b) Incorrect answers are associated with a higher frequency of underthinking than correct ones. (Underthinking threshold L=100 tokens; "R1-Distill" is DeepSeek-R1-Distill-Qwen).
• Figure 3: The overall pipeline of the SmartSwitch Inference Framework. During generation, the Perception module monitors for thought switches. When a switch occurs, the preceding thought is evaluated by a Process Reward Model (PRM). If $T_{k-1}$ is deemed promising (score above threshold), the Intervention module activates: generation is paused, the context is reverted to the end of $T_{k-1}$, a "deepen prompt" is inserted, and generation resumes, encouraging deeper exploration of $T_{k-1}$. If not promising, the generation continues.
• Figure 4: SmartSwitch reduces underthinking frequency and the number of thought-switches on AIME24. "R1-Distill" abbreviates "DeepSeek-R1-Distill-Qwen".
• Figure 5: Pseudocode of the SmartSwitch inference algorithm. The framework continuously monitors the generated token stream for thought-switch cues. Upon detection, the preceding thought is evaluated by a Process Reward Model (PRM). If its score exceeds a threshold ($\tau_{\text{score}}$), generation is interrupted and redirected to further explore the promising thought via a "deepen prompt." Otherwise, the model proceeds with the new thought without intervention.