Are Your Reasoning Models Reasoning or Guessing? A Mechanistic Analysis of Hierarchical Reasoning Models

TL;DR

Are Your Reasoning Models Reasoning or Guessing? performs a mechanistic analysis of Hierarchical Reasoning Models (HRM) on Sudoku-Extreme to determine whether HRMs truly reason or merely guess. By tracing latent trajectories, the authors identify fixed-point violations, grokking dynamics, and multiple fixed points, arguing that HRM effectively behaves as a search over latent states rather than incremental reasoning. They propose three scaling strategies—data augmentation, input perturbation, and model bootstrapping—and demonstrate that Augmented HRM achieves up to $96.9\%$ accuracy on Sudoku-Extreme, significantly surpassing prior HRM variants. The work offers a framework for diagnosing recursive reasoning systems and lays out concrete techniques to align their search dynamics with genuine reasoning, improving both understanding and performance.

Abstract

Hierarchical reasoning model (HRM) achieves extraordinary performance on various reasoning tasks, significantly outperforming large language model-based reasoners. To understand the strengths and potential failure modes of HRM, we conduct a mechanistic study on its reasoning patterns and find three surprising facts: (a) Failure of extremely simple puzzles, e.g., HRM can fail on a puzzle with only one unknown cell. We attribute this failure to the violation of the fixed point property, a fundamental assumption of HRM. (b) "Grokking" dynamics in reasoning steps, i.e., the answer is not improved uniformly, but instead there is a critical reasoning step that suddenly makes the answer correct; (c) Existence of multiple fixed points. HRM "guesses" the first fixed point, which could be incorrect, and gets trapped there for a while or forever. All facts imply that HRM appears to be "guessing" instead of "reasoning". Leveraging this "guessing" picture, we propose three strategies to scale HRM's guesses: data augmentation (scaling the quality of guesses), input perturbation (scaling the number of guesses by leveraging inference randomness), and model bootstrapping (scaling the number of guesses by leveraging training randomness). On the practical side, by combining all methods, we develop Augmented HRM, boosting accuracy on Sudoku-Extreme from 54.5% to 96.9%. On the scientific side, our analysis provides new insights into how reasoning models "reason".

Figures (7)

• Figure 1: A lone unknown cell exposes the fixed-point violation. HRM secretly guesses fixed points, no matter they are true or not. Multiple fixed points exist in the latent space; escaping them via data augmentation, input and model bootstrapping boosts accuracy from 54.5% to 96.9%.
• Figure 2: Reasoning trajectories in latent space (projected onto first two principal components) for different sudoku puzzles, and associated answers (red cell: wrong, green cell: correct). (a) for a difficult sudoku from the validation set of Sudoku-Extreme, the answer is correct; (b) For a simple puzzle with only one row masked, the answer is correct for the first two segments but continues updating to wrong answers, violating the fixed point assumption; (c) Complete failure on an extremely simple puzzle with only one masked token.
• Figure 3: Data mixing restores stability and symmetry of latent reasoning trajectories (projected onto the first two principal components). It eliminates unfavored drifts after getting the correct answer. Furthermore, when dealing with distinct samples simplified via the same format, all latent trajectories now show perfect symmetry.
• Figure 4: Segment-wise loss scaling improves over training. When averaging across the test samples, more segments lead to smaller losses in a smooth and gradual way, contrasting the "grokking" curves for per-sample analysis in Figure \ref{['single_loss']}.
• Figure 5: Per-sample analysis shows "grokking" dynamics along segments. For success samples, the loss value hovers above some threshold value before suddenly dropping to zero.