Understanding Jailbreak Success: A Study of Latent Space Dynamics in Large Language Models

TL;DR

<p>Confronting the challenge that jailbreaks can bypass safety safeguards in large language models, the paper develops a mechanistic, latent-space analysis across multiple model families to understand how jailbreaks operate. It constructs contrastive jailbreak steering vectors from activation differences, demonstrates cross-class transferability, and investigates whether jailbreaks work by suppressing the model's perception of prompt harmfulness. The results show consistent, transferable vectors that mitigate jailbreak success across classes and reveal a harmfulness-suppression signal that is stronger in some models, suggesting a shared internal mechanism with practical implications for defense. Overall, the work lays groundwork for robust jailbreak countermeasures and a deeper mechanistic understanding of latent-space dynamics in LLMs.</p>

Abstract

Conversational large language models are trained to refuse to answer harmful questions. However, emergent jailbreaking techniques can still elicit unsafe outputs, presenting an ongoing challenge for model alignment. To better understand how different jailbreak types circumvent safeguards, this paper analyses model activations on different jailbreak inputs. We find that it is possible to extract a jailbreak vector from a single class of jailbreaks that works to mitigate jailbreak effectiveness from other semantically-dissimilar classes. This may indicate that different kinds of effective jailbreaks operate via a similar internal mechanism. We investigate a potential common mechanism of harmfulness feature suppression, and find evidence that effective jailbreaks noticeably reduce a model's perception of prompt harmfulness. These findings offer actionable insights for developing more robust jailbreak countermeasures and lay the groundwork for a deeper, mechanistic understanding of jailbreak dynamics in language models.

Figures (18)

• Figure 1: Example of steering with the jailbreak vector prefix_injection to prevent the jailbreak success of another jailbreak type (GCG). Steering is performed on Qwen1.5 14B Chat at layer 20 with multiplier $-1$.
• Figure 2: PCA on activation differences between harmful requests with and without the jailbreak. Activations are extracted at last instruction token position in the middle layer of the models.
• Figure 3: Cosine similarity scores between jailbreak steering vectors.
• Figure 4: Example of jailbreak vector steering with the refusal_suppression vector preventing a successful jailbreak from a different class (payload_split). Steering is performed on Vicuna 13B v1.5 at layer 20 with multiplier $-1$.
• Figure 5: PCA on last instruction token activations for harmful and harmless questions, Vicuna 13B, layer 20.