A Closer Look at Adversarial Suffix Learning for Jailbreaking LLMs: Augmented Adversarial Trigger Learning

TL;DR

ATLA reframes adversarial trigger learning for jailbreaking LLMs by replacing the vanilla NLL objective with a weighted loss that emphasizes response-format tokens, coupled with an auxiliary I-awareness loss to suppress evasive outputs. This combination enables learning from a single (Q,R) pair, strong generalization to new queries, and transferability to unseen models, while maintaining training efficiency. Empirical evaluations on system-prompt leakage and jailbreaking across multiple LLMs show near-perfect attack success and substantially reduced query costs, with robust ablations and cross-model transfer. The work also demonstrates how ATLA can be extended (e.g., ATLA-K) and composed with other jailbreak methods, highlighting both the potential attack surface and implications for defense research.

Abstract

Gradient optimization-based adversarial attack methods automate the learning of adversarial triggers to generate jailbreak prompts or leak system prompts. In this work, we take a closer look at the optimization objective of adversarial trigger learning and propose ATLA: Adversarial Trigger Learning with Augmented objectives. ATLA improves the negative log-likelihood loss used by previous studies into a weighted loss formulation that encourages the learned adversarial triggers to optimize more towards response format tokens. This enables ATLA to learn an adversarial trigger from just one query-response pair and the learned trigger generalizes well to other similar queries. We further design a variation to augment trigger optimization with an auxiliary loss that suppresses evasive responses. We showcase how to use ATLA to learn adversarial suffixes jailbreaking LLMs and to extract hidden system prompts. Empirically we demonstrate that ATLA consistently outperforms current state-of-the-art techniques, achieving nearly 100% success in attacking while requiring 80% fewer queries. ATLA learned jailbreak suffixes demonstrate high generalization to unseen queries and transfer well to new LLMs. We released our code https://github.com/QData/ALTA_Augmented_Adversarial_Trigger_Learning

Figures (24)

• Figure 1: Visualizing ATLA weighted loss for every token in a target response ${R}$. We also show each token's $\textbf{NLL}$ loss as reference bars. ATLA weighted loss formulation guides the attack optimization to pay even more attention to format-related tokens.
• Figure 2: We collect the token level's $\textbf{NLL}$ loss from 50 different target responses from Llama2-7b-chat . For all (token, loss) pairs, we rank the tokens according to the loss values, and visualize the first 25 tokens with highest loss (aka also highest weights in ATLA${\mathcal{L}^e}$). (Full figure see Fig. \ref{['fig:loss_distirbution']}).
• Figure 3: Comparing the query costs of the adversarial suffix learning from GCG, ATLA, and ATLA's two ablations, which are ATLA w/o ${\mathcal{L}^e}$ and ATLA w/o ${\mathcal{L}^s}$, on different pretrained LLMs. The $x$-axis represents the LLM’s forward budget, the $y$-axis represents attack success rate: the proportion of the suffixes that can bypass the safeguard and perform a successful attack.
• Figure 4: Log-Perplexity distributions for suffixes learned with ATLA and GCG. We learn adversarial suffixes against Llama2-7b-chat and Vicuna-7b-1.5 , and then evaluate the log-perplexity of the corresponding input prompts.
• Figure 5: Comparing the generalization ability of the learned adversarial suffixes when facing new harmful questions. We learn 100 adversarial suffixes for each of the four methods: GCG, ATLA, ATLA w/o ${\mathcal{L}^e}$, and ATLA w/o ${\mathcal{L}^s}$, and group them into different G-ASR bins.

Theorems & Definitions (4)

• Theorem 1
• Lemma 1: nemirovski2009robust & sagawa2019distributionally
• Definition 1
• Definition 2