Steering Large Language Models using Conceptors: Improving Addition-Based Activation Engineering

TL;DR

This paper tackles reliably steering LLM outputs by activation engineering and introduces conceptor steering, a soft-projection approach that replaces simple additive vectors with ellipsoidal representations computed from activation covariances, i.e., $C = R (R + α^{-2} I)^{-1}$. Steering is applied as $h' = β_c C h$, enabling more precise control over activation patterns and supporting Boolean combinations of steering targets. The method is demonstrated on function-vector tasks using GPT-J and GPT-NeoX, where conceptor steering consistently outperforms additive steering, and composite steering is enabled via AND operations over conceptors. While incurring offline computation and a new hyperparameter α, conceptor steering offers region-based activation control with potential implications for debiasing and safety, and the authors provide open-source code for reproducibility.

Abstract

Large language models have transformed AI, yet reliably controlling their outputs remains a challenge. This paper explores activation engineering, where outputs of pre-trained LLMs are controlled by manipulating their activations at inference time. Unlike traditional methods using a single steering vector, we introduce conceptors - mathematical constructs that represent sets of activation vectors as ellipsoidal regions. Conceptors act as soft projection matrices and offer more precise control over complex activation patterns. Our experiments demonstrate that conceptors outperform traditional methods across multiple steering tasks. We further use Boolean operations on conceptors for combined steering goals that empirically outperform additively combining steering vectors on a set of tasks. These results highlight conceptors as a promising tool for more effective steering of LLMs. Our code is available on github.com/jorispos/conceptorsteering.

Figures (14)

• Figure 1: Illustration showing the basic geometric difference between additive and conceptor steering using a set of activations for the antonym task. Additive steering acts as a translation of the activation vectors by a fixed steering vector. Conceptor steering acts as a (soft) projection onto a target ellipsoid.
• Figure 2: Illustration of three conceptors as ellipsoids that capture the state space region of different sets of neural activations in 3D space (black points). Reproduced from Jaeger jaeger2014conceptorseasyintroduction.
• Figure 3: Comparison of the accuracy on all six function tasks for conceptor-based steering against additive steering across all layers for GPT-J and GPT-NeoX. For explanation, see main text.
• Figure 4: The effect of mean centering on conceptor-based and addition-based steering on the GPT-J (6B) model across all layers, computed on five different function vector tasks (% accuracy). The line shows the best average performance across five runs for the best hyperparameters for the given layer.
• Figure 5: Performance of additive steering and conceptor steering on composite functions. For explanation of the figure caption, see text. Dashed lines represent the "baseline" where the steering mechanism is computed on the composite task. Solid lines show task arithmetic.