Graph of Attacks with Pruning: Optimizing Stealthy Jailbreak Prompt Generation for Enhanced LLM Content Moderation

TL;DR

GAP introduces a graph-based framework for adversarial jailbreak prompt generation that enables knowledge sharing across attack paths, dramatically improving attack success rates and reducing query costs compared with tree-based methods like TAP. By maintaining a global context and sequential seed learning, GAP achieves efficient exploration and robust performance across text-only and multimodal targets, with GAP-Auto and GAP-VLM extending applicability. The approach also demonstrates practical value by augmenting content moderation systems through data generation for guardrail training, yielding substantial improvements in detection metrics. Overall, GAP represents a significant step toward more reliable and scalable LLM safety evaluation and defense enhancement.

Abstract

As large language models (LLMs) become increasingly prevalent, ensuring their robustness against adversarial misuse is crucial. This paper introduces the GAP (Graph of Attacks with Pruning) framework, an advanced approach for generating stealthy jailbreak prompts to evaluate and enhance LLM safeguards. GAP addresses limitations in existing tree-based LLM jailbreak methods by implementing an interconnected graph structure that enables knowledge sharing across attack paths. Our experimental evaluation demonstrates GAP's superiority over existing techniques, achieving a 20.8% increase in attack success rates while reducing query costs by 62.7%. GAP consistently outperforms state-of-the-art methods for attacking both open and closed LLMs, with attack success rates of >96%. Additionally, we present specialized variants like GAP-Auto for automated seed generation and GAP-VLM for multimodal attacks. GAP-generated prompts prove highly effective in improving content moderation systems, increasing true positive detection rates by 108.5% and accuracy by 183.6% when used for fine-tuning. Our implementation is available at https://github.com/dsbuddy/GAP-LLM-Safety.

Figures (8)

• Figure 1: Comparing TAP and GAP attack strategies across four sequential seed prompts. The top row shows TAP, where each seed independently generates a full attack tree in its own color, maintaining consistent tree sizes due to no knowledge sharing between iterations. The bottom row demonstrates GAP, where mixed-colored nodes indicate reuse of successful vulnerability patterns from previous seeds, enabling knowledge transfer across sequential iterations. This knowledge sharing in GAP results in progressively smaller and more efficient trees from left to right, as redundant refinements become unnecessary. By the fourth seed, GAP exhibits a notably streamlined structure compared to TAP, indicating successful attack path optimization through accumulated knowledge.
• Figure 2: GAP vs TAP Performance Across Target Models. Vulnerability detection success rates for GAP-M (green circles), GAP-V (blue squares), and TAP (red triangles) against increasing query budgets across three different target models, demonstrating GAP variants' consistent superior performance and efficiency.
• Figure 3: Two-phase framework for automated generation of diverse and fine-grained prompts. Phase 1 uses metaprompting with Mistral-123B-v2407 to expand categories into behaviors. Phase 2 generates balanced harmful and benign prompts for comprehensive evaluation.
• Figure 4: Attacker policy system message used throughout the GAP-Auto seed generation process.
• Figure 5: Meta prompt for decomposing high-level content policy categories into specific fine-grained behaviors.