When Can Large Reasoning Models Save Thinking? Mechanistic Analysis of Behavioral Divergence in Reasoning

TL;DR

This work probes the internal mechanics of RL-trained large reasoning models when prompted to save thinking, revealing three distinct thinking modes: No Thinking (NT), Explicit Thinking (ET), and Implicit Thinking (IT). By analyzing termination confidence, attention from thinking to generation, and input-section focus on GSM8K and MATH500, the study links internal states to observed efficiency-accuracy trade-offs. NT yields substantial reductions in output length but notably lowers accuracy, whereas ET and IT preserve or improve accuracy while shortening responses. The findings expose fundamental inconsistencies in RL-optimized reasoning and point to the need for adaptive prompting or training strategies to achieve reliable, efficient reasoning in LRMs.

Abstract

Large reasoning models (LRMs) have significantly advanced performance on complex tasks, yet their tendency to overthink introduces inefficiencies. This study investigates the internal mechanisms of reinforcement learning (RL)-trained LRMs when prompted to save thinking, revealing three distinct thinking modes: no thinking (NT), explicit thinking (ET), and implicit thinking (IT). Through comprehensive analysis of confidence in thinking termination, attention from thinking to generation, and attentional focus on input sections, we uncover key factors influencing the reasoning behaviors. We further find that NT reduces output length at the cost of accuracy, while ET and IT maintain accuracy with reduced response length. Our findings expose fundamental inconsistencies in RL-optimized LRMs, necessitating adaptive improvements for reliable efficiency.

Figures (7)

• Figure 1: Examples of the three modes of QwQ-32B under the save-thinking instructions. The final answers are underlined. The thought is marked in italics. The token </think> marks the end of the thought.
• Figure 2: PCA visualization of attention activation from the last layer for all samples in GSM8K and MATH500.
• Figure 3: Davies-Bouldin Index (less is better) calculated for the NT and ET clusters based on layer-wise attention activation across all 64 layers.
• Figure 4: The density distribution of attention scores under different thinking modes. Implicit thinking samples in MATH500 are too few to plot a density curve. The green dashed line indicates their average value.
• Figure 5: Attention scores across prompt sections for different thinking modes on the MATH500 dataset. The results on GSM8K are given in Appendix \ref{['appx:summed_attention_gsm8k']} and similar findings can be obtained.