Reasoning Beyond Chain-of-Thought: A Latent Computational Mode in Large Language Models

TL;DR

Reasoning Beyond Chain-of-Thought investigates whether a latent internal mechanism underpins multi-step reasoning in LLMs, rather than relying exclusively on explicit CoT prompts. The authors identify reasoning-related latent features with Sparse Autoencoders and validate causality by steering a singleton feature using an intervention strength $\alpha$, injecting residual activations at the first generation step across six models and three benchmarks. Results show that steered direct prompts can match or surpass CoT performance, often with shorter reasoning traces, and can even override instructions like no_think, suggesting CoT is a trigger but not the sole cause of reasoning. These findings reveal a latent reasoning mode in LLMs and point to activation-level interventions as a practical path for robust, controllable reasoning.

Abstract

Chain-of-Thought (CoT) prompting has improved the reasoning performance of large language models (LLMs), but it remains unclear why it works and whether it is the unique mechanism for triggering reasoning in large language models. In this work, we study this question by directly analyzing and intervening on the internal representations of LLMs with Sparse Autoencoders (SAEs), identifying a small set of latent features that are causally associated with LLM reasoning behavior. Across multiple model families and reasoning benchmarks, we find that steering a single reasoning-related latent feature can substantially improve accuracy without explicit CoT prompting. For large models, latent steering achieves performance comparable to standard CoT prompting while producing more efficient outputs. We further observe that this reasoning-oriented internal state is triggered early in generation and can override prompt-level instructions that discourage explicit reasoning. Overall, our results suggest that multi-step reasoning in LLMs is supported by latent internal activations that can be externally activated, while CoT prompting is one effective, but not unique, way of activating this mechanism rather than its necessary cause.

Figures (15)

• Figure 1: Multiple triggers for latent reasoning in LLMs.Top: CoT prompting produces explicit reasoning text, while the internal mechanism responsible for its effectiveness remains unclear. Bottom: We view reasoning as a latent internal mechanism that can be activated through different triggers, including latent steering.
• Figure 2: Overview of the proposed two-stage pipeline. (a) Feature Discovery. We contrast direct and chain-of-thought prompting to extract token-level activations, project them into sparse latent features using a pretrained sparse autoencoder (SAE), and identify prompt-sensitive candidate features via differential analysis. (b) Causal Validation. We apply targeted latent steering to selected features and inject the resulting residual into the model to assess their intervention sensitivity, evaluating the effect on model behavior and answer correctness.
• Figure 3: Behaviors induced by steering. (a) Steering can override prompt-level instructions that suppress reasoning (e.g., \\ no_think). (b) For questions requiring multi-step computation, steering can yield shorter explicit reasoning traces compared to standard chain-of-thought prompting. (c) On retrieval-heavy questions, latent steering leads the model to explicitly note that step-by-step reasoning is unnecessary. These examples are illustrative and not intended to be exhaustive.
• Figure 4: Activation dynamics of a reasoning-related SAE feature during generation on GSM8K.
• Figure 5: Effect of intervention timing when steering reasoning-related latent features on GSM8K with LLaMA-3.1-8B-Instruct. The figure reports accuracy (left) and generated token length (right) as a function of the decoding step at which the intervention is applied.