Steering Safely or Off a Cliff? Rethinking Specificity and Robustness in Inference-Time Interventions

TL;DR

This work interrogates the safety and precision of inference-time steering for LLMs by formalizing a three-dimensional specificity framework (general, control, robust) and applying it to two safety-critical tasks: overrefusal and faithfulness hallucinations. It demonstrates that while steering can achieve high efficacy and preserve broad capabilities and related control properties, robustness specificity deteriorates under distribution shifts and adversarial jailbreaks, undermining safety in practical use. The study benchmarks five steering methods across multiple models and datasets, revealing a consistent trade-off: improvements in target behavior often accompany increased vulnerability to adversarial contexts. The authors advocate evaluating steering not only on efficacy but also on robustness and specificity, provide a detailed experimental framework, and highlight the need for methods that generalize safely beyond in-distribution settings. These findings bear on the deployment of steering in real-world systems, emphasizing cautious optimization and robust evaluation to avoid unintended compromises in safety.

Abstract

Model steering, which involves intervening on hidden representations at inference time, has emerged as a lightweight alternative to finetuning for precisely controlling large language models. While steering efficacy has been widely studied, evaluations of whether interventions alter only the intended property remain limited, especially with respect to unintended changes in behaviors related to the target property. We call this notion specificity. We propose a framework that distinguishes three dimensions of specificity: general (preserving fluency and unrelated abilities), control (preserving related control properties), and robustness (preserving control properties under distribution shifts). We study two safety-critical use cases: steering models to reduce overrefusal and faithfulness hallucinations, and show that while steering achieves high efficacy and largely maintains general and control specificity, it consistently fails to preserve robustness specificity. In the case of overrefusal steering, for example, all steering methods reduce overrefusal without harming general abilities and refusal on harmful queries; however, they substantially increase vulnerability to jailbreaks. Our work provides the first systematic evaluation of specificity in model steering, showing that standard efficacy and specificity checks are insufficient, because without robustness evaluation, steering methods may appear reliable even when they compromise model safety.

Figures (5)

• Figure 1: Refusal and overrefusal behavior in LLMs (top left). Existing evaluations predominantly focus on efficacy (bottom left)---does steering reduce overrefusal. We extend evaluation of steering methods to assess specificity across three dimensions (right)---does steering preserve unrelated abilities (general), preserve refusal on harmful queries (control), and remain safe under adversarial prompts (robustness)?
• Figure 2: Faithfulness hallucination steering evaluation.
• Figure 3: Utility and safety trade-off for different steering methods. The arrows indicate the difference between safety in-distribution vs out-of-distribution. Steering improves the ComplianceRate on pseudo-harmful queries with some drop in safety. Importantly, jailbreaking success is higher after steering compared to baseline and steering methods with larger gains in utility generally show a lower robustness to adversarial attacks.
• Figure 4: Top 2 principal components of activations at the last token position in layer 20 for harmless, pseudo-harmful, harmful, and harmful queries with jailbreaking prefix. The dotted lines visualize the decision boundaries corresponding to the different steering vectors. Activations of harmful queries with jailbreak prefixes lie markedly closer to that of harmless queries.
• Figure 5: First two principal components of hidden activations (at the last token position in layer 20) for harmless, pseudo-harmful, harmful, and harmful queries with jailbreaking prefix. The dotted lines visualize the decision boundaries corresponding to the different steering vectors. Adding jailbreaking prompts to harmful queries shifts hidden activations towards those of safe queries.