LLM-VA: Resolving the Jailbreak-Overrefusal Trade-off via Vector Alignment

TL;DR

LLMs face a safety challenge: they may jailbreak by answering harmful prompts or over-refuse benign ones. The authors show that the model internally encodes separate decision vectors for answering ($v_a$) and input safety ($v_b$), which are nearly orthogonal, making magnitude-based steering unable to resolve the jailbreak-overrefusal trade-off. They propose LLM-VA, a vector-alignment approach that uses closed-form weight updates to align $v_a$ with $v_b$ across selected layers, thereby making the willingness to answer depend on safety judgments without fine-tuning or architectural changes. Across 12 models, LLM-VA improves F1 by 11.45% over the best baseline while preserving 95.92% of utility, and adapts automatically to each model's safety bias, offering a scalable and effective safety enhancement for diverse LLMs.

Abstract

Safety-aligned LLMs suffer from two failure modes: jailbreak (answering harmful inputs) and over-refusal (declining benign queries). Existing vector steering methods adjust the magnitude of answer vectors, but this creates a fundamental trade-off -- reducing jailbreak increases over-refusal and vice versa. We identify the root cause: LLMs encode the decision to answer (answer vector $v_a$) and the judgment of input safety (benign vector $v_b$) as nearly orthogonal directions, treating them as independent processes. We propose LLM-VA, which aligns $v_a$ with $v_b$ through closed-form weight updates, making the model's willingness to answer causally dependent on its safety assessment -- without fine-tuning or architectural changes. Our method identifies vectors at each layer using SVMs, selects safety-relevant layers, and iteratively aligns vectors via minimum-norm weight modifications. Experiments on 12 LLMs demonstrate that LLM-VA achieves 11.45% higher F1 than the best baseline while preserving 95.92% utility, and automatically adapts to each model's safety bias without manual tuning. Code and models are available at https://hotbento.github.io/LLM-VA-Web/.

Figures (12)

• Figure 1: The angles between answer vectors ($v_a$) and benign vectors ($v_b$) are approximately $90^\circ$ across layers in gemma-2-9b-it, indicating near-orthogonality between answer decisions and safety assessments.
• Figure 2: The distributions of the projections onto the benign, answer vectors at different layers of Llama-3.1-8B-Instruct. The left, middle, right figures correspond to the 4th, 16th, and 28th MLP layers, respectively.
• Figure 3: Unlike existing methods that only adjust the magnitude of $v_a$ (trading off jailbreak vs. over-refusal), LLM-VA aligns $v_a$ with $v_b$ to address both issues.
• Figure 4: The framework of LLM-VA.
• Figure 5: Left: Average utility preservation by method. Right: Utility preservation per LLM with LLM-VA. Values near 1.0 indicate minimal degradation.