HarmTransform: Transforming Explicit Harmful Queries into Stealthy via Multi-Agent Debate

TL;DR

HarmTransform introduces a multi-agent debate framework to transform explicit harmful queries into stealthier forms while preserving their underlying malicious intent, addressing a gap in LLM safety alignment for covert harms. The method combines persona-based multi-agent debate, a summarization step, and a generation step to produce $Q_{IMP}$ from $Q_{EXP}$, with a data-quality evaluation framework balancing intent preservation and evasion effectiveness. Empirical results show HarmTransform outperforms baselines on attacking effectiveness and maintains competitive intent preservation, while ablations reveal complexity and topic shifts can arise with more agents or rounds. The work highlights both the potential of multi-agent debate to enrich safety training data and its limitations, offering practical guidance for future data-generation and safety-robustness research.

Abstract

Large language models (LLMs) are equipped with safety mechanisms to detect and block harmful queries, yet current alignment approaches primarily focus on overtly dangerous content and overlook more subtle threats. However, users can often disguise harmful intent through covert rephrasing that preserves malicious objectives while appearing benign, which creates a significant gap in existing safety training data. To address this limitation, we introduce HarmTransform, a multi-agent debate framework for systematically transforming harmful queries into stealthier forms while preserving their underlying harmful intent. Our framework leverages iterative critique and refinement among multiple agents to generate high-quality, covert harmful query transformations that can be used to improve future LLM safety alignment. Experiments demonstrate that HarmTransform significantly outperforms standard baselines in producing effective query transformations. At the same time, our analysis reveals that debate acts as a double-edged sword: while it can sharpen transformations and improve stealth, it may also introduce topic shifts and unnecessary complexity. These insights highlight both the promise and the limitations of multi-agent debate for generating comprehensive safety training data.

Figures (12)

• Figure 1: Overview of the HarmTransform pipeline. The framework consists of three main components: (1) multi-agent debating, (2) argument summarization, and (3) query transformation into a stealthy form.
• Figure 2: Example of personas for role-playing setup. The full list of personas can be found in Appendix \ref{['app:Personas']}
• Figure 3: An example where the transformation process leads to a purely benign query, causing loss of harmful intent.
• Figure 4: HarmTransform performance under different numbers of debaters with the debate rounds fixed to 1.
• Figure 5: HarmTransform performance under different numbers of debating rounds with the number of debaters fixed to 3. Round 0 indicates the initial statement without debating.