Extracting Latent Steering Vectors from Pretrained Language Models

TL;DR

This work demonstrates that controllable text generation can be achieved by extracting latent steering vectors directly from a frozen GPT-2 decoder, obviating fine-tuning. By optimizing a fixed-length vector per target sentence and injecting it at middle transformer layers, the method attains near-perfect recovery across diverse domains. It further shows that steering vectors support unsupervised sentiment transfer with competitive performance and enable meaningful semantic similarity via vector distances, while revealing a rich, interpretable latent geometry through interpolation and robustness analyses. The findings suggest that a frozen LM's latent space can be harnessed for targeted generation and style manipulation with limited or no additional training. The approach offers a new lens on controllable generation and raises important considerations regarding biases and responsible usage.

Abstract

Prior work on controllable text generation has focused on learning how to control language models through trainable decoding, smart-prompt design, or fine-tuning based on a desired objective. We hypothesize that the information needed to steer the model to generate a target sentence is already encoded within the model. Accordingly, we explore a different approach altogether: extracting latent vectors directly from pretrained language model decoders without fine-tuning. Experiments show that there exist steering vectors, which, when added to the hidden states of the language model, generate a target sentence nearly perfectly (> 99 BLEU) for English sentences from a variety of domains. We show that vector arithmetic can be used for unsupervised sentiment transfer on the Yelp sentiment benchmark, with performance comparable to models tailored to this task. We find that distances between steering vectors reflect sentence similarity when evaluated on a textual similarity benchmark (STS-B), outperforming pooled hidden states of models. Finally, we present an analysis of the intrinsic properties of the steering vectors. Taken together, our results suggest that frozen LMs can be effectively controlled through their latent steering space.

Figures (6)

• Figure 1: Our approach adds a vector ${\boldsymbol{z}_{steer}}$ to the activations of a pretrained transformer decoder to steer it to decode a desired target sentence. We experiment with adding ${\boldsymbol{z}_{steer}}$ to different locations inside a GPT-2 model at different timesteps. Experiments reveal that our approach can recover sequences nearly perfectly and that injecting the steering vector in the middle layers of the transformer stack performs best. Layer normalizations and residual connections inside the transformer block are omitted for clarity.
• Figure 2: TSNE projection of 8 steering vectors initialized from different random seeds for 20 different sentences (injected at layer 6, after self-attention). ${\boldsymbol{z}_{steer}}$ is well-separated for different sentences, and the different seeds are tightly clustered for the same target sentence, indicating that our extraction method is robust.
• Figure 3: Evaluation of unsupervised sentiment transfer on the Yelp dataset. The plot shows accuracy vs. self-BLEU by varying $\lambda=$ (0.25, 0.5, 1.0, 1.5, 2.0, 2.5, 3.0, 4.0, 5.0, 10.0) for our method. Overall, the steering vectors perform comparably to prior work.
• Figure 4: On the test split of STS-B, we measure Spearman rank correlation ($\rho \cdot100$) between sentence similarity scores and cosine similarities between the steering vectors extracted from GPT2-117M when injected at different layers at the first timestep for those sentences. The vertical lines indicate extractive baselines: mean-pooled final hidden states for GPT2-117M and BERT-base as well as mean-pooled GloVe vectors. Results show that extracted steering vectors outperform these.
• Figure 5: Interpolation between steering vectors extracted from two pairs of random sentences from the Yelp Sentiment test set. Decoding from interpolated vectors from two sentences produces well-formed output that incrementally changes the sentiment and meaning.