Scaling Reasoning Hop Exposes Weaknesses: Demystifying and Improving Hop Generalization in Large Language Models

TL;DR

The paper investigates why chain-of-thought reasoning in large language models collapses as the number of reasoning hops increases. By adopting an error-centric view and residual-stream analysis, the authors reveal a competition mechanism where erroneous processing heads (ep heads) can bias intermediate reasoning away from correct trajectories, which are mediated by cp heads and aw heads. They show that errors concentrate on a small set of token-level error types and that deleting specific ep heads often restores correct predictions, allowing a lightweight test-time intervention, TCR, to dynamically deactivate these heads during decoding. TCR uses a head selector to identify candidate ep heads and an entropy-based detector to trigger interventions, yielding consistent improvements across seven tasks and four open-source LLMs, with larger gains when error localization is precise (TCR-gold). Overall, the work identifies actionable internal mechanisms behind hop-generalization failures and proposes a practical, generalizable remedy to improve long-horizon reasoning in LLMs.

Abstract

Chain-of-thought (CoT) reasoning has become the standard paradigm for enabling Large Language Models (LLMs) to solve complex problems. However, recent studies reveal a sharp performance drop in reasoning hop generalization scenarios, where the required number of reasoning steps exceeds training distributions while the underlying algorithm remains unchanged. The internal mechanisms driving this failure remain poorly understood. In this work, we conduct a systematic study on tasks from multiple domains, and find that errors concentrate at token positions of a few critical error types, rather than being uniformly distributed. Closer inspection reveals that these token-level erroneous predictions stem from internal competition mechanisms: certain attention heads, termed erroneous processing heads (ep heads), tip the balance by amplifying incorrect reasoning trajectories while suppressing correct ones. Notably, removing individual ep heads during inference can often restore the correct predictions. Motivated by these insights, we propose test-time correction of reasoning, a lightweight intervention method that dynamically identifies and deactivates ep heads in the reasoning process. Extensive experiments across different tasks and LLMs show that it consistently improves reasoning hop generalization, highlighting both its effectiveness and potential.

Figures (37)

• Figure 1: Illustration of the reasoning circuit and the competition mechanism inside LLMs.
• Figure 2: Overall accuracy (green curve) and error proportion for key error types (blue bar) and other error types (purple bar) of Qwen2.5-7B-Instruct on Parity-NL, MDM and ObjC.
• Figure 3: Illustration of the idea of comparative study and constructing the correct prediction set $S_{\text{corr}}$ and erroneous prediction set $S_{\text{err}}$ with the multi-digit multiplication (MDM) task (error type 2, digit decomposition error). Each sample contains the input reasoning context (i.e., the concatenation of input problem and part of the model generated response) and the output predicted token. $S_{\text{corr}}$ and $S_{\text{err}}$ contains the samples that are correctly and erroneously predicted by model, respectively. In this paper, we study the reasoning hop generalization mechanism through comparatively studying the mechanism behind these correct and erroneous predictions.
• Figure 4: Locating answer-writing heads with our proposed measure for $S_{\text{corr}}$ and $S_{\text{err}}$ and tracing ground-truth and predicted tokens' information across intermediate layers.
• Figure 5: Decoding the information in aw heads: $\mathbf{a}_{1}^{22}$ and $\mathbf{a}_{11}^{23}$. (a,c) and (b,d) show their $\text{top}10$ decoded tokens before (i.e., the original model) and after knocking out the processing head $\mathbf{a}_{7}^{0}$.