Amplified Vulnerabilities: Structured Jailbreak Attacks on LLM-based Multi-Agent Debate

TL;DR

The paper investigates jailbreak vulnerabilities in Large Language Model–based Multi-Agent Debate (MAD) systems, revealing that MAD dynamics inherently amplify unsafe outputs compared to single-agent setups. It introduces a novel structured prompt rewriting framework that employs Narrative Encapsulation, Role-Driven Escalation, Iterative Refinement, and Rhetorical Obfuscation to trigger harmful content across four MAD frameworks (Multi-Persona, Exchange of Thoughts, ChatEval, AgentVerse) and four base models (GPT-4o, GPT-4, GPT-3.5-turbo, DeepSeek). Experimental results show substantial increases in Process Harmfulness Score, Answer Harmfulness Score, and Harmfulness Diffusion Rate, with attack success rates reaching up to 80% in some configurations, especially when underlying models are less safety-aligned. The work emphasizes the need for MAD-specific defenses, such as intra-debate monitoring, safety-oriented evaluators, robust persona design, and adversarial training, to ensure safe deployment of MAD systems in real-world tasks. Overall, the study highlights critical security gaps in current MAD architectures and provides a concrete attack framework and evaluative methodology to guide future defense development.

Abstract

Multi-Agent Debate (MAD), leveraging collaborative interactions among Large Language Models (LLMs), aim to enhance reasoning capabilities in complex tasks. However, the security implications of their iterative dialogues and role-playing characteristics, particularly susceptibility to jailbreak attacks eliciting harmful content, remain critically underexplored. This paper systematically investigates the jailbreak vulnerabilities of four prominent MAD frameworks built upon leading commercial LLMs (GPT-4o, GPT-4, GPT-3.5-turbo, and DeepSeek) without compromising internal agents. We introduce a novel structured prompt-rewriting framework specifically designed to exploit MAD dynamics via narrative encapsulation, role-driven escalation, iterative refinement, and rhetorical obfuscation. Our extensive experiments demonstrate that MAD systems are inherently more vulnerable than single-agent setups. Crucially, our proposed attack methodology significantly amplifies this fragility, increasing average harmfulness from 28.14% to 80.34% and achieving attack success rates as high as 80% in certain scenarios. These findings reveal intrinsic vulnerabilities in MAD architectures and underscore the urgent need for robust, specialized defenses prior to real-world deployment.

Figures (5)

• Figure 1: Comparison of jailbreak attempts on MAD. A standard harmful query is rejected by single-agent and standard MAD setups through coordinated refusal. However, the same query, when processed by our prompt rewriting template, overcomes these refusals, enabling successful jailbreaks and exposing MAD vulnerabilities. Detailed results of this demo are given in Appendix \ref{['detailed debate result of demo']}.
• Figure 2: An example of the agent roles and workflow of different MAD models.
• Figure 3: Maximum harmfulness score in each debate round on Harmful Generation without jailbreak rewriting. AgentVerse tends to get the final answer after one round, so we do not show its results.
• Figure 4: Maximum harmfulness score in each debate round on Harmful Generation with jailbreak rewriting. AgentVerse tends to get the final answer after one round, so we do not show its results.
• Figure 5: A comparison of ASR before and after the jailbreak prompt is rewritten.