Towards Understanding Steering Strength

TL;DR

This paper provides the first theoretical analysis of steering strength in activation steering for LLMs by framing a difference-of-means steering vector within a tractable Unconstrained Features Model. It derives how the steering strength $\alpha$ influences next-token probabilities, concept presence in the output, and cross-entropy, revealing a non-monotonic bump in token probabilities, a sigmoidal rise in target-concept presence, and a locally quadratic degradation of cross-entropy near $\alpha=0$ followed by plateauing at large $|\alpha|$. The authors validate these predictions across eleven language models and extend the framework to real-world decoder-only transformers, showing consistent qualitative behaviors and identifying a practical steering 'sweet spot' for balancing efficacy with output quality. The work provides principled guidance for choosing $\alpha$ and highlights avenues for adaptive prompting and broader steering methods in deployment contexts.

Abstract

A popular approach to post-training control of large language models (LLMs) is the steering of intermediate latent representations. Namely, identify a well-chosen direction depending on the task at hand and perturbs representations along this direction at inference time. While many propositions exist to pick this direction, considerably less is understood about how to choose the magnitude of the move, whereas its importance is clear: too little and the intended behavior does not emerge, too much and the model's performance degrades beyond repair. In this work, we propose the first theoretical analysis of steering strength. We characterize its effect on next token probability, presence of a concept, and cross-entropy, deriving precise qualitative laws governing these quantities. Our analysis reveals surprising behaviors, including non-monotonic effects of steering strength. We validate our theoretical predictions empirically on eleven language models, ranging from a small GPT architecture to modern models.

Figures (21)

• Figure 1: Top: Constructing a steering vector $\mathbf{v}$, for the target concept "code safety", at the $\ell^{\text{th}}$ block. We run two contrastive prompt sets ($n_+$ safe and $n_-$ malicious) through an $L$-block LLM and collect the representations $\{{\color{neg} \mathbf{h}_-}, {\color{pos} \mathbf{h}_+}\}$ at layer $\ell$ for each prompt. Averaging these representations over all safe prompts gives $\color{pos} \mu_+$, and for the malicious prompts it gives $\color{neg}\mu_-$ (both marked by a cross). We then define $\mathbf{v} \vcentcolon = \mu_+ - \mu_-$. Bottom: Steering the model’s response toward safe behavior on a new prompt is done by adding $\color{violet}\alpha \mathbf{v}$ to the residual stream $\mathbf{h}$ at $\ell^{\text{th}}$ block. The steering strength $\alpha$ controls how far representations are moved along $v$.
• Figure 2: Visualization of dataset next-token probabilities $(p(z \mid \mathbf{c}_j))_{z \in [V]}$ for the vocabulary of Eq. \ref{['eq:dataset']}: probabilities for the context $\mathbf{c}_2 = {\color{blue}aab}$ are shown in solid-color, while probabilities for $\mathbf{c}_1 = {\color{orange}ABB}$ are shown transparent. This illustrates our dataset condition ${\color{blue}a_z }> {\color{blue!70}b_z}$: a token is more likely when it belongs to the same concept as the context, which is why the solid-color blue points lie above their transparent counterparts.
• Figure 3: Next-token probability increases $\Delta p(\alpha)$ for a fixed context. Each curve corresponds to a token $z$: target tokens $\mathcal{T}$ are in blue and off-target tokens in orange. Most target tokens exhibit a "bump" (peaking at $\alpha_{(1,1)}$), while one target token increases and off-target tokens decrease.
• Figure 4: Concept probability increases $\Delta p(\mathcal{C} \mid \alpha)$ predicted by Th. \ref{['th:increasing-concept']}: the target concept $\color{blue}\Delta p(\mathcal{T} \mid \alpha)$ increases with a sigmoidal shape, an off-target $\color{pink!90!violet}\Delta p(\mathcal{C} \mid \alpha)$ decreases sigmoidally, and another $\color{orange}\Delta p(\mathcal{C}' \mid \alpha)$ converges to the same limit as $|\alpha|\to\infty$.
• Figure 5: Local quadratic behavior of $\Delta \mathrm{CE}(\alpha)$, as predicted by Thm. \ref{['th:sweet-spot']}. The blue curve shows $\Delta \mathrm{CE}(\alpha)$ and the black curve the quadratic fit using the coefficient from the theorem.

Theorems & Definitions (22)

• Definition 2.1: Dataset next-token probabilities
• Definition 2.2: Unconstrained Features Model
• Definition 3.1: Probability increase
• Definition 3.2: Log-odds
• Theorem 3.3: Behavior of $\Delta p$
• Remark 3.4: Sign of $\alpha_{(j,z)}$
• Definition 3.5: Increase/decrease of a concept
• Theorem 3.6: Behavior of $\Delta p(\mathcal{C} \mid \alpha)$
• Definition 3.7: Difference of cross-entropy
• Theorem 3.8: Cross-entropy local behavior