Siren: A Learning-Based Multi-Turn Attack Framework for Simulating Real-World Human Jailbreak Behaviors

TL;DR

Siren, a learning-based multi-turn attack framework designed to simulate real-world human jailbreak behaviors, is proposed and it is hoped Siren inspires the development of stronger defenses against advanced multi-turn jailbreak attacks under realistic scenarios.

Abstract

Large language models (LLMs) are widely used in real-world applications, raising concerns about their safety and trustworthiness. While red-teaming with jailbreak prompts exposes the vulnerabilities of LLMs, current efforts focus primarily on single-turn attacks, overlooking the multi-turn strategies used by real-world adversaries. Existing multi-turn methods rely on static patterns or predefined logical chains, failing to account for the dynamic strategies during attacks. We propose Siren, a learning-based multi-turn attack framework designed to simulate real-world human jailbreak behaviors. Siren consists of three stages: (1) MiniMax-driven training set construction utilizing Turn-Level LLM feedback, (2) post-training attackers with supervised fine-tuning (SFT) and direct preference optimization (DPO), and (3) interactions between the attacking and target LLMs. Experiments demonstrate that Siren achieves an attack success rate (ASR) of 90% with LLaMA-3-8B as the attacker against Gemini-1.5-Pro as the target model, and 70% with Mistral-7B against GPT-4o, significantly outperforming single-turn baselines. Moreover, Siren with a 7B-scale model achieves performance comparable to a multi-turn baseline that leverages GPT-4o as the attacker, while requiring fewer turns and employing decomposition strategies that are better semantically aligned with attack goals. We hope Siren inspires the development of stronger defenses against advanced multi-turn jailbreak attacks under realistic scenarios. Code is available at https://github.com/YiyiyiZhao/siren. Warning: This paper contains potentially harmful text.

Figures (4)

• Figure 1: Jailbreak Attack Strategies. (a) Use a single adversarial query to bypass the target LLM's safety mechanisms, e.g., GCG zou2023universaltransferableadversarialattacks. (b) Iteratively refine adversarial prompts via attacker-LLM interactions, e.g., PAIR chao2024jailbreakingblackboxlarge. (c) Multi-turn attacks like ActorAttack ren2024derailyourselfmultiturnllm, Crescendo russinovich2024greatwritearticlethat, and 'Speak Out of Turn' zhou2024speakturnsafetyvulnerability follow predefined patterns to decompose queries, with the attacking LLM frozen. (d) Siren (Ours): A learning-based approach where the attacking LLM is post-trained based on the target LLM's responses.
• Figure 2: Overview of the Siren Framework. (1) Training Set Construction: Turn-level feedback from a sophisticated LLM (e.g., GPT-4o) is used to generate training samples. At each turn $i$, the adversarial goal $x_{\text{goal}}$ and the historical interaction context $(x^{(1)}, r^{(1)}, \dots, x^{(i-1)}, r^{(i-1)})$ are the inputs, and the output of the attacking LLM is the attack query $x^{(i)}$. (2) Post-Training: The attacking LLM is fine-tuned through SFT and DPO, with negative outputs generated by a relatively weak LLM (e.g., Baichuan-13B). (3) Interaction Phase: The attacking LLM interacts with the target LLM over multiple turns, dynamically generating queries with a maximum of 4 turns, typically up to 3 turns. Through iterations over time, new interaction records can be further utilized to fine-tune the attacking LLM for red-teaming purposes. The training set is constructed using the DAN FobiddenQuestions dataset, and the interactions are applied to the HarmfulBehaviors benchmark, ensuring no dataset overlap through a deduplication process.
• Figure 3: Training set construction. Step 1: Adversarial questions are decomposed into multi-turn queries using strategies and expanded through LLM-augmented generation. Step 2: The target LLM answers each query based on prior interactions. Step 3: Each response is labeled as either Harmless or Harmful using LLM-assisted evaluation. Step 4: Turn-level samples with harmful responses are selected and deduplicated for training the adversarial LLM.
• Figure 4: Similarity Distribution: Semantic similarity between the original attack goal and its decomposed queries is markedly higher when employing Siren.