Uncovering Latent Chain of Thought Vectors in Language Models

TL;DR

The paper tackles inducing Chain-of-Thought reasoning in language models without prompting by perturbing the activation space at inference. It derives a CoT-oriented steering vector $\mathbf{v}$ from contrasts $\mathbf{v}(q) = \mathbb{E}_t [\mathbf{a}^{(l)}_t(q_{\text{CoT}})] - \mathbb{E}_t [\mathbf{a}^{(l)}_t(q_{\text{direct}})]$ and aggregates it as $\mathbf{v} = \mathbb{E}_{q \sim \mathcal{D}} [\mathbf{v}(q)]$, then injects $c\,\mathbf{v}$ into a LM's residual stream via a PyTorch hook. Empirical results on Llama3 8B Instruct and Mistral 7B v0.2 Instruct show competitive or superior performance to traditional CoT prompting across GSM8k, MMLU, ARC-AI2, and AGI-Eval, with generalization to SAT Math despite not being in the steering data. The findings imply that reasoning patterns may be encoded in activation- and weight-space, enabling a lightweight, inference-time approach to tune model behavior, with injection timing and strategy significantly impacting outcomes.

Abstract

In this work, we examine how targeted perturbations in the activation space of Language Models (LMs) can encode complex reasoning patterns. We inject steering vectors, derived from LM activations, into LMs during inference time and study whether these vectors can induce Chain-of-Thought (CoT) reasoning in LMs without the need for natural language prompting. We demonstrate this approach on Llama3 8B Instruct and Mistral 7B v0.2 Instruct and show that activation-space interventions achieve competitive, if not superior, performance compared to traditional CoT prompting across multiple reasoning benchmarks, including GSM8k, MMLU, AGI Eval, and ARC AI2. These findings suggest that neural network activations can encode reasoning patterns, offering a new application of activation space manipulation as a tool for tuning model behavior.

Figures (3)

• Figure 1: Overview of Steering Method Vector Creation Workflow -- we use two contrasting natural language prompts alongside questions from reasoning benchmarks to extract "Think Step by Step" and "Answer Immediately" layer residual stream activations from the LM. We then average these activations across token positions and across a corpus of reasoning questions, subtracting the two vectors to obtain a proposed "CoT Steering Vector" for injection.
• Figure 2: Llama3 8B Instruct: Steered vs. Unsteered Response
• Figure 3: Mistral 7B v0.2 Instruct: Steered vs. Baseline Response