Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Probing and Steering Evaluation Awareness of Language Models

Jord Nguyen, Khiem Hoang, Carlo Leonardo Attubato, Felix Hofstätter

TL;DR

The paper investigates evaluation awareness in Llama-3.3-70B-Instruct and introduces linear probes to reveal internally encoded distinctions between evaluation and deployment contexts. It demonstrates that deception and safety evaluation prompts are often perceived as test-like by the model, raising concerns about the realism and validity of benchmarks. Steering experiments show mixed results, with simple suffix prompts outperforming linear-probe steering in countering sandbagging, underscoring the need for robust, live evaluation frameworks. Overall, the work provides a framework for white-box safety audits and emphasizes improving evaluation design to ensure trustworthy model governance.

Abstract

Language models can distinguish between testing and deployment phases -- a capability known as evaluation awareness. This has significant safety and policy implications, potentially undermining the reliability of evaluations that are central to AI governance frameworks and voluntary industry commitments. In this paper, we study evaluation awareness in Llama-3.3-70B-Instruct. We show that linear probes can separate real-world evaluation and deployment prompts, suggesting that current models internally represent this distinction. We also find that current safety evaluations are correctly classified by the probes, suggesting that they already appear artificial or inauthentic to models. Our findings underscore the importance of ensuring trustworthy evaluations and understanding deceptive capabilities. More broadly, our work showcases how model internals may be leveraged to support blackbox methods in safety audits, especially for future models more competent at evaluation awareness and deception.

Probing and Steering Evaluation Awareness of Language Models

TL;DR

The paper investigates evaluation awareness in Llama-3.3-70B-Instruct and introduces linear probes to reveal internally encoded distinctions between evaluation and deployment contexts. It demonstrates that deception and safety evaluation prompts are often perceived as test-like by the model, raising concerns about the realism and validity of benchmarks. Steering experiments show mixed results, with simple suffix prompts outperforming linear-probe steering in countering sandbagging, underscoring the need for robust, live evaluation frameworks. Overall, the work provides a framework for white-box safety audits and emphasizes improving evaluation design to ensure trustworthy model governance.

Abstract

Language models can distinguish between testing and deployment phases -- a capability known as evaluation awareness. This has significant safety and policy implications, potentially undermining the reliability of evaluations that are central to AI governance frameworks and voluntary industry commitments. In this paper, we study evaluation awareness in Llama-3.3-70B-Instruct. We show that linear probes can separate real-world evaluation and deployment prompts, suggesting that current models internally represent this distinction. We also find that current safety evaluations are correctly classified by the probes, suggesting that they already appear artificial or inauthentic to models. Our findings underscore the importance of ensuring trustworthy evaluations and understanding deceptive capabilities. More broadly, our work showcases how model internals may be leveraged to support blackbox methods in safety audits, especially for future models more competent at evaluation awareness and deception.

Paper Structure

This paper contains 23 sections, 9 figures.

Table of Contents

  1. Introduction
  2. Related Work
  3. Situational Awareness and Evaluation Awareness in Language Models
  4. Alignment, Deception, and Sandbagging
  5. Linear Representations and Interpretability
  6. Methodology
  7. Model and Data
  8. Probe Training
  9. Prompt Stages Classification
  10. Sandbagging Steering
  11. Results
  12. Evaluation Awareness Probe Performance
  13. Evaluating Prompt Realism with Probe
  14. Quantitative Analysis
  15. Qualitative Analysis
  16. ...and 8 more sections

Figures (9)

  • Figure 1: Diagram of linear probe training, adapted from rimsky-etal-2024-steering
  • Figure 2: Probe performance using AUROC scores on SAD stages_ oversight
  • Figure 3: Simple Contrastive probe AUROC score compared to baselines on SAD stages_ oversight
  • Figure 4: Probe generalisation performance on validation datasets, using optimal classification threshold from SAD stages_ oversight. Simple Contrastive probe performs best.
  • Figure 5: Probe classifies the majority of safety evaluation prompts as test-like. Alpaca dataset is displayed as control.
  • ...and 4 more figures