Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Patterns and Mechanisms of Contrastive Activation Engineering

Yixiong Hao, Ayush Panda, Stepan Shabalin, Sheikh Abdur Raheem Ali

TL;DR

This work investigates contrastive activation engineering (CAE) as a zero-cost, inference-time method to steer LLM outputs by injecting a direction in hidden-state space derived from the mean difference between desired and undesired activations, formalized as $A'_l(x) = A_l(x) + \alpha ( A_l(x_+)[-1] - A_l(x_-)[-1] )$ (and its dataset-averaged variant). It analyzes CAE across in-distribution and out-of-distribution settings, showing reliable steering primarily within the distribution used to compute the steering vectors, with diminishing returns after roughly 100 samples and noticeable degradation in perplexity. The study also finds CAE is vulnerable to adversarial prompts via Evolutionary Prompt Optimization (EPO) and that larger models tend to be more resistant to steering-induced degradation, while out-of-distribution performance remains a major challenge. The paper concludes with practical guidelines and future directions, including improving OOD robustness, automating data collection, and extending CAE to multi-behavior steering and additional CAE techniques.

Abstract

Controlling the behavior of Large Language Models (LLMs) remains a significant challenge due to their inherent complexity and opacity. While techniques like fine-tuning can modify model behavior, they typically require extensive computational resources. Recent work has introduced a class of contrastive activation engineering (CAE) techniques as promising approaches for steering LLM outputs through targeted modifications to their internal representations. Applied at inference-time with zero cost, CAE has the potential to introduce a new paradigm of flexible, task-specific LLM behavior tuning. We analyze the performance of CAE in in-distribution, out-of-distribution settings, evaluate drawbacks, and begin to develop comprehensive guidelines for its effective deployment. We find that 1. CAE is only reliably effective when applied to in-distribution contexts. 2. Increasing the number of samples used to generate steering vectors has diminishing returns at around 80 samples. 3. Steering vectors are susceptible to adversarial inputs that reverses the behavior that is steered for. 4. Steering vectors harm the overall model perplexity. 5. Larger models are more resistant to steering-induced degradation.

Patterns and Mechanisms of Contrastive Activation Engineering

TL;DR

This work investigates contrastive activation engineering (CAE) as a zero-cost, inference-time method to steer LLM outputs by injecting a direction in hidden-state space derived from the mean difference between desired and undesired activations, formalized as (and its dataset-averaged variant). It analyzes CAE across in-distribution and out-of-distribution settings, showing reliable steering primarily within the distribution used to compute the steering vectors, with diminishing returns after roughly 100 samples and noticeable degradation in perplexity. The study also finds CAE is vulnerable to adversarial prompts via Evolutionary Prompt Optimization (EPO) and that larger models tend to be more resistant to steering-induced degradation, while out-of-distribution performance remains a major challenge. The paper concludes with practical guidelines and future directions, including improving OOD robustness, automating data collection, and extending CAE to multi-behavior steering and additional CAE techniques.

Abstract

Controlling the behavior of Large Language Models (LLMs) remains a significant challenge due to their inherent complexity and opacity. While techniques like fine-tuning can modify model behavior, they typically require extensive computational resources. Recent work has introduced a class of contrastive activation engineering (CAE) techniques as promising approaches for steering LLM outputs through targeted modifications to their internal representations. Applied at inference-time with zero cost, CAE has the potential to introduce a new paradigm of flexible, task-specific LLM behavior tuning. We analyze the performance of CAE in in-distribution, out-of-distribution settings, evaluate drawbacks, and begin to develop comprehensive guidelines for its effective deployment. We find that 1. CAE is only reliably effective when applied to in-distribution contexts. 2. Increasing the number of samples used to generate steering vectors has diminishing returns at around 80 samples. 3. Steering vectors are susceptible to adversarial inputs that reverses the behavior that is steered for. 4. Steering vectors harm the overall model perplexity. 5. Larger models are more resistant to steering-induced degradation.
Paper Structure (20 sections, 2 equations, 64 figures, 2 tables)

This paper contains 20 sections, 2 equations, 64 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Background and related works
  3. Theory of steering.
  4. Steering vector injection.
  5. Evaluating steering.
  6. CAE formulation
  7. In-distribution analysis
  8. Out-of-distribution analysis
  9. Perplexity of steering
  10. Red teaming steering vectors with EPO
  11. Limitations
  12. Conclusion and extensions
  13. Datasets
  14. In-distribution - extended graphs
  15. Layer Sweep
  16. ...and 5 more sections

Figures (64)

  • Figure 1: Contrastive activation engineering
  • Figure 2: Llama 8B sweep across layers with strengths +1 and -1. Increasing line opacity represents the percentage of MWE used to compute steering vector in this order: (20, 40, 60, 80, 100) . Layer 15 is optimal for Llama 8B.
  • Figure 3: Llama 8B sweep across steering strengths. Line opacity encodes the number of samples. Percentage of answer matching behavior increase for strengths between 0 and +2 but decreases otherwise.
  • Figure 4: Llama 70B sweep across layers with strengths +1 and -1. Increasing line opacity represents the percentage of MWE used to compute steering vector in this order: (20, 40, 60, 80, 100). Layer 29 is optimal for Llama 70B
  • Figure 5: Llama 70B sweep across steering strengths. Line opacity encodes the number of samples. Percentage of answer matching behavior increase for strengths between 0 and +6 but decreases otherwise.
  • ...and 59 more figures