Reason2Attack: Jailbreaking Text-to-Image Models via LLM Reasoning

TL;DR

Reason2Attack (R2A) tackles safety vulnerabilities in text-to-image models by training an LLM to autonomously generate adversarial prompts through a Frame Semantics–driven CoT synthesis and a two-stage reasoning pipeline. The first stage fine-tunes the LLM on CoT examples to internalize a robust adversarial reasoning process, while the second stage uses reinforcement learning with an attack process reward that jointly optimizes prompt stealthiness, effectiveness, and length. The attack process reward addresses the sparse feedback problem and guides efficient exploration, yielding higher attack success with fewer queries and strong transferability to both open-source and commercial T2I systems. Empirical results demonstrate that R2A outperforms baselines in ASR, reduces query requirements, and transfers across diverse models, underscoring real-world safety risks and the need for stronger defenses.

Abstract

Text-to-Image(T2I) models typically deploy safety filters to prevent the generation of sensitive images. Unfortunately, recent jailbreaking attack methods manually design instructions for the LLM to generate adversarial prompts, which effectively bypass safety filters while producing sensitive images, exposing safety vulnerabilities of T2I models. However, due to the LLM's limited understanding of the T2I model and its safety filters, existing methods require numerous queries to achieve a successful attack, limiting their practical applicability. To address this issue, we propose Reason2Attack(R2A), which aims to enhance the LLM's reasoning capabilities in generating adversarial prompts by incorporating the jailbreaking attack into the post-training process of the LLM. Specifically, we first propose a CoT example synthesis pipeline based on Frame Semantics, which generates adversarial prompts by identifying related terms and corresponding context illustrations. Using CoT examples generated by the pipeline, we fine-tune the LLM to understand the reasoning path and format the output structure. Subsequently, we incorporate the jailbreaking attack task into the reinforcement learning process of the LLM and design an attack process reward that considers prompt length, prompt stealthiness, and prompt effectiveness, aiming to further enhance reasoning accuracy. Extensive experiments on various T2I models show that R2A achieves a better attack success ratio while requiring fewer queries than baselines. Moreover, our adversarial prompts demonstrate strong attack transferability across both open-source and commercial T2I models.

Figures (10)

• Figure 1: The framework of Reason2Attack (R2A). First, we introduce a unified generation framework based on Frame Semantics, which generates CoT adversarial prompts in a step-by-step manner. Second, we present a two-stage LLM reasoning training framework guided by the attack process. In the first stage, the LLM is fine-tuned with CoT examples generated by the unified framework to internalize the adversarial prompt generation process grounded in Frame Semantics. In the second stage, we incorporate jailbreaking attacks into the LLM’s reinforcement learning and propose an attack process reward that uses diverse feedback signals, enabling the LLM to understand the black-box T2I model and safety mechanisms.
• Figure 1: The prompt template for the LLM to generate an adversarial prompt based on the inputted sensitive prompt.
• Figure 2: A jailbreaking attack example generated by R2A. Images are generated by SDV3.
• Figure 2: The return curve of reinforcement learning in our LLM reasoning training. (a) Training LLM using only reinforcement learning with the attack process reward. (b) Training LLM using only reinforcement learning with our proposed attack process reward. (c) Training LLM first via the supervised fine-tuning process with the CoT examples generated from the unified generation framework, followed by reinforcement learning with the attack process reward.
• Figure 3: Visualization of attack results using R2A across DALL$\cdot$E 3 and Midjourney. Generated images are blurred for display.

Theorems & Definitions (1)

• Definition 1: Jailbreaking attack on T2I models via LLM reasoning