Understanding and Mitigating Over-refusal for Large Language Models via Safety Representation

TL;DR

This work addresses the problem of over-refusal in safety-aligned large language models by analyzing it through the lens of internal representations. It introduces MOSR, a defense with two components: overlap-aware loss weighting to reduce erasure of benign prompts that resemble over-refusal and context-aware augmentation to provide richer context before rejection decisions. Empirical results show MOSR improves the safety-over-refusal balance across multiple models and datasets, while largely preserving general capabilities; ablations confirm the individual and combined value of its components. The findings suggest that future jailbreak defenses should consider representation-space dynamics to maintain safety without unduly sacrificing usability, and point to multi-turn and cost-efficient strategies as promising directions.

Abstract

Large language models demonstrate powerful capabilities across various natural language processing tasks, yet they also harbor safety vulnerabilities. To enhance LLM safety, various jailbreak defense methods have been proposed to guard against harmful outputs. However, improvements in model safety often come at the cost of severe over-refusal, failing to strike a good balance between safety and usability. In this paper, we first analyze the causes of over-refusal from a representation perspective, revealing that over-refusal samples reside at the boundary between benign and malicious samples. Based on this, we propose MOSR, designed to mitigate over-refusal by intervening the safety representation of LLMs. MOSR incorporates two novel components: (1) Overlap-Aware Loss Weighting, which determines the erasure weight for malicious samples by quantifying their similarity to pseudo-malicious samples in the representation space, and (2) Context-Aware Augmentation, which supplements the necessary context for rejection decisions by adding harmful prefixes before rejection responses. Experiments demonstrate that our method outperforms existing approaches in mitigating over-refusal while largely maintaining safety. Overall, we advocate that future defense methods should strike a better balance between safety and over-refusal.

Figures (11)

• Figure 1: LLM Over-refusal phenomenon illustration: LLMs reject malicious user requests while also rejecting benign ones.
• Figure 2: Performance of existing defense methods in terms of safety versus over-refusal.
• Figure 3: Flowchart illustrating the process of understanding over-refusal through representation. It includes three steps: Step 1 trains a probe classifier with LLM hidden states to verify its ability to distinguish safe and malicious prompts. Step 2 uses the pre-trained classifier to detect if the model misclassifies over-refusal samples as malicious. Step 3 applies LogitLens to decode layer-wise representations of different prompts, revealing the model's tone tendency and semantic perception differences.
• Figure 4: Probe test results. (a) test results on benign and malicious samples; (b) test results on the over-refusal set.
• Figure 5: Decoding results for benign, harmful, and over-refusal prompts.