Decoding the Critique Mechanism in Large Reasoning Models

Abstract

Large Reasoning Models (LRMs) exhibit backtracking and self-verification mechanisms that enable them to revise intermediate steps and reach correct solutions, yielding strong performance on complex logical benchmarks. We hypothesize that such behaviors are beneficial only when the model has sufficiently strong "critique" ability to detect its own mistakes. This work systematically investigates how current LRMs recover from errors by inserting arithmetic mistakes in their intermediate reasoning steps. Notably, we discover a peculiar yet important phenomenon: despite the error propagating through the chain-of-thought (CoT), resulting in an incorrect intermediate conclusion, the model still reaches the correct final answer. This recovery implies that the model must possess an internal mechanism to detect errors and trigger self-correction, which we refer to as the hidden critique ability. Building on feature space analysis, we identify a highly interpretable critique vector representing this behavior. Extensive experiments across multiple model scales and families demonstrate that steering latent representations with this vector improves the model's error detection capability and enhances the performance of test-time scaling at no extra training cost. Our findings provide a valuable understanding of LRMs' critique behavior, suggesting a promising direction to control and improve their self-verification mechanism. Our code is available at https://github.com/mail-research/lrm-critique-vectors.

Figures (15)

• Figure 1: Hidden self-correction despite erroneous reasoning in R1-32B. Left: original correct reasoning. Right: injected error (3 + 4 = 6) propagates to incorrect thinking conclusion ($20), yet the model recovers to the correct final answer ($18). We hypothesize this indicates implicit error detection beyond the observable CoT. Full generation details are in Appendix~\ref{['sec:generation_example_app']}.
• Figure 2: Distribution of reasoning-result alignment across R1 and Qwen3 model variants. The proportions represent four distinct outcomes based on the correctness of the internal "thinking" process versus the correctness of the final "answer" on the GSM8K-Error and MATH500-Error benchmarks. In this paper, we analyze the self-correction mechanisms represented by the light blue segments: '$\times$ Think $\checkmark$ Answer'.
• Figure 3: Effect of different steering coefficients $\alpha$ evaluated on ProcessBench. Left to right: Error detection accuracy, correct solution accuracy, and F1 score. Increasing $\alpha$ generally improves error detection and F1 scores while leading to a decline in correct solution accuracy. The dashed vertical line indicates the baseline performance at $\alpha=0$.
• Figure 4: Effect of different steering coefficients $\alpha$ evaluated on BIG-Bench Mistake. Left to right: Error detection accuracy, correct solution accuracy, and F1 score. The results illustrate similar effects to Figure \ref{['fig:process_bench_results']}, where there is a monotonic relationship between $\alpha$ and error detection. The dashed vertical line indicates the baseline performance at $\alpha=0$.
• Figure 5: Qualitative example of positive steering on BIG-Bench arithmetic task with R1-32B. There is a subtle arithmetic error introduced in the prompt ($7+2+20=30$, whereas it should be $29$). While the baseline model hallucinates that the math is correct during verification, positive steering enables the model to successfully catch the addition error.