Toward Understanding the Transferability of Adversarial Suffixes in Large Language Models

TL;DR

The paper tackles the transferability of adversarial suffixes that jailbreak large language models by performing a large-scale statistical and interventional study of activation-space geometry. It identifies three mechanistic factors—refusal connectivity, suffix push, and orthogonal shift—that correlate with transfer success, while showing semantic similarity between prompts is only weakly predictive. Through intra- and inter-model analyses and targeted interventions, it demonstrates that suffix push is the strongest predictor of transfer and that prompting and loss-function adjustments can modulate attack success. The findings offer actionable insights for both designing more transferable attacks and developing defenses that mitigate suffix-based jailbreaks by shaping the relevant activation-space directions.

Abstract

Discrete optimization-based jailbreaking attacks on large language models aim to generate short, nonsensical suffixes that, when appended onto input prompts, elicit disallowed content. Notably, these suffixes are often transferable -- succeeding on prompts and models for which they were never optimized. And yet, despite the fact that transferability is surprising and empirically well-established, the field lacks a rigorous analysis of when and why transfer occurs. To fill this gap, we identify three statistical properties that strongly correlate with transfer success across numerous experimental settings: (1) how much a prompt without a suffix activates a model's internal refusal direction, (2) how strongly a suffix induces a push away from this direction, and (3) how large these shifts are in directions orthogonal to refusal. On the other hand, we find that prompt semantic similarity only weakly correlates with transfer success. These findings lead to a more fine-grained understanding of transferability, which we use in interventional experiments to showcase how our statistical analysis can translate into practical improvements in attack success.

Figures (13)

• Figure 1: Intra-model transfer with one suffix per prompt for different models. Cells are colored when a suffix successfully jailbreaks a prompt.
• Figure 2: Intra-model transfer with multiple suffixes per prompt. Cells are colored when the corresponding suffix of the seed jailbreaks a prompt.
• Figure 3: Inter-model transfer between Llama 3.2 and Qwen. Cells are colored when a suffix successfully jailbreaks a prompt.
• Figure 4: Distribution spread comparison of cosine similarity and (normalized) dot product activations with refusal direction across models.
• Figure 5: Cross-layer suppression of refusal direction by most and least powerful suffixes for different models, figure based on arditi2024refusal. Activations are taken at the end-of-instruction token.

Theorems & Definitions (7)

• Definition 1: Attack success rate (ASR)
• Definition 2: Refusal direction arditi2024refusal
• Definition 3: Semantic similarity
• Definition 4: Refusal connectivity
• Definition 5: Suffix push
• Definition 6: Orthogonal shift
• Definition 7: Optimal layer selection