SCANS: Mitigating the Exaggerated Safety for LLMs via Safety-Conscious Activation Steering

TL;DR

SCANS proposes Safety-Conscious Activation Steering to mitigate exaggerated safety in safety-aligned LLMs by extracting refusal vectors from activation space and steering middle-layer representations. It anchors safety-critical layers via vocabulary projection and PCA, and uses a similarity-based classifier to determine steering direction for each input, enabling a balance between preventing harmful refusals and preserving helpful responses. Empirical results across Llama2 and Vicuna models show significant reductions in false refusals on XSTest/OKTest with minimal degradation to general capabilities, at only minor inference/memory overhead. The work contributes a training-free, representation-engineering approach to safety alignment and suggests further exploration of activation-space steering for robust alignment.

Abstract

Safety alignment is indispensable for Large Language Models (LLMs) to defend threats from malicious instructions. However, recent researches reveal safety-aligned LLMs prone to reject benign queries due to the exaggerated safety issue, limiting their helpfulness. In this paper, we propose a Safety-Conscious Activation Steering (SCANS) method to mitigate the exaggerated safety concerns in aligned LLMs. First, SCANS extracts the refusal steering vectors within the activation space and utilizes vocabulary projection to anchor some specific safety-critical layers which influence model refusal behavior. Second, by tracking the hidden state transition, SCANS identifies the steering direction and steers the model behavior accordingly, achieving a balance between exaggerated safety and adequate safety. Experiments show that SCANS achieves new state-of-the-art performance on XSTest and OKTest benchmarks, without impairing their defense capability against harmful queries and maintaining almost unchanged model capability.

Figures (6)

• Figure 1: One example of exaggerated safety phenomenon in aligned LLMs. Here, 'kill' means turning off the light without malicious intention but the original model makes a false refusal. With SCANS, the model generates helpful response.
• Figure 2: The overview of SCANS, which extracts the refusal behavior vectors, and then determines the steering direction and steers the model behavior, thereby guaranteeing adequate safety without exaggerating safety.
• Figure 3: t-SNE visualization of hidden state transition on XSTest dataset at layers 9, 20 and 32 of Llama2-7b-chat. The results indicate safety-related representation clustering emerges in middle and latter layers.
• Figure 4: Classification performance of $\sigma(q)$ and all baselines on XSTest dataset. Llama Guard, GradSafe and SCANS-$\sigma(q)$ are all based on Llama2-7b-chat model.
• Figure 5: Cosine similarity (in terms of hidden state transition) with the unsafe reference of each layer for XSTest dataset based on Llama2-7b-chat.