Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Towards Reliable Evaluation of Behavior Steering Interventions in LLMs

Itamar Pres, Laura Ruis, Ekdeep Singh Lubana, David Krueger

TL;DR

This work advocates for four properties missing from current evaluations of representation engineering methods, and introduces an evaluation pipeline grounded in these criteria, offering both a quantitative and visual analysis of how effectively a given method works.

Abstract

Representation engineering methods have recently shown promise for enabling efficient steering of model behavior. However, evaluation pipelines for these methods have primarily relied on subjective demonstrations, instead of quantitative, objective metrics. We aim to take a step towards addressing this issue by advocating for four properties missing from current evaluations: (i) contexts sufficiently similar to downstream tasks should be used for assessing intervention quality; (ii) model likelihoods should be accounted for; (iii) evaluations should allow for standardized comparisons across different target behaviors; and (iv) baseline comparisons should be offered. We introduce an evaluation pipeline grounded in these criteria, offering both a quantitative and visual analysis of how effectively a given method works. We use this pipeline to evaluate two representation engineering methods on how effectively they can steer behaviors such as truthfulness and corrigibility, finding that some interventions are less effective than previously reported.

Towards Reliable Evaluation of Behavior Steering Interventions in LLMs

TL;DR

This work advocates for four properties missing from current evaluations of representation engineering methods, and introduces an evaluation pipeline grounded in these criteria, offering both a quantitative and visual analysis of how effectively a given method works.

Abstract

Representation engineering methods have recently shown promise for enabling efficient steering of model behavior. However, evaluation pipelines for these methods have primarily relied on subjective demonstrations, instead of quantitative, objective metrics. We aim to take a step towards addressing this issue by advocating for four properties missing from current evaluations: (i) contexts sufficiently similar to downstream tasks should be used for assessing intervention quality; (ii) model likelihoods should be accounted for; (iii) evaluations should allow for standardized comparisons across different target behaviors; and (iv) baseline comparisons should be offered. We introduce an evaluation pipeline grounded in these criteria, offering both a quantitative and visual analysis of how effectively a given method works. We use this pipeline to evaluate two representation engineering methods on how effectively they can steer behaviors such as truthfulness and corrigibility, finding that some interventions are less effective than previously reported.

Paper Structure

This paper contains 12 sections, 2 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Desiderata when Evaluating Activation Steering Methods
  3. Methodology
  4. Experiments
  5. Discussion and Conclusion
  6. Acknowledgements
  7. Link to Code
  8. Limitations
  9. Related Work
  10. Experimental Motivation for Property 1
  11. Experimental Motivation for Property 2
  12. Experiment Details

Figures (2)

  • Figure 1: Proposed evaluation pipeline. (a) A prompt designed to elicit behavioral preferences has both a behavior matching and mismatching continuation appended to it. The model evaluates these samples with and without the intervention applied, recording likelihoods for each. (b) Likelihood visualization showing intervention effectiveness. Ideally, the intervention reduces negative sample likelihoods and increases positive sample likelihoods.
  • Figure 2: Behavioral steering evaluations. Each panel shows renormalized likelihoods (LL) of behavior-matching (positive) and mismatching (negative) continuations under baseline and intervened models. Ideal interventions lower negative and raise positive likelihoods relative to baseline. The top 25% most likely negative samples and least likely positive samples are highlighted.