GhostPrompt: Jailbreaking Text-to-image Generative Models based on Dynamic Optimization

TL;DR

GhostPrompt tackles the vulnerability of text-to-image systems to NSFW content by addressing modern, semantics-aware safety filters. It introduces a two-pronged approach: dynamic prompt optimization that iteratively refines adversarial prompts using text-safety feedback and CLIP alignment, and adaptive safety indicator injection that employs reinforcement-learning to insert benign visual cues in images. The method achieves state-of-the-art bypass performance across multiple text and image safety filters, with high semantic fidelity (CLIP score ~0.276) and strong generalization to unseen systems like GPT-4.1 and DALL·E 3, while also delivering improved efficiency (fewer queries). These results reveal systemic vulnerabilities in current multimodal defenses and establish GhostPrompt as a valuable controlled red-teaming tool for strengthening safety pipelines in vision-language models.

Abstract

Text-to-image (T2I) generation models can inadvertently produce not-safe-for-work (NSFW) content, prompting the integration of text and image safety filters. Recent advances employ large language models (LLMs) for semantic-level detection, rendering traditional token-level perturbation attacks largely ineffective. However, our evaluation shows that existing jailbreak methods are ineffective against these modern filters. We introduce GhostPrompt, the first automated jailbreak framework that combines dynamic prompt optimization with multimodal feedback. It consists of two key components: (i) Dynamic Optimization, an iterative process that guides a large language model (LLM) using feedback from text safety filters and CLIP similarity scores to generate semantically aligned adversarial prompts; and (ii) Adaptive Safety Indicator Injection, which formulates the injection of benign visual cues as a reinforcement learning problem to bypass image-level filters. GhostPrompt achieves state-of-the-art performance, increasing the ShieldLM-7B bypass rate from 12.5\% (Sneakyprompt) to 99.0\%, improving CLIP score from 0.2637 to 0.2762, and reducing the time cost by $4.2 \times$. Moreover, it generalizes to unseen filters including GPT-4.1 and successfully jailbreaks DALLE 3 to generate NSFW images in our evaluation, revealing systemic vulnerabilities in current multimodal defenses. To support further research on AI safety and red-teaming, we will release code and adversarial prompts under a controlled-access protocol.

Figures (12)

• Figure 1: Overview of the pipeline for generating adversarial prompts to attack both text and image safety filters in text-to-image (T2I) models.
• Figure 2: Overall pipeline of GhostPrompt.
• Figure 3: Left: Target vs. adversarial prompt; Right: Effect of adaptive safety indicator injection. On the left, the target prompt is blocked by safety filters, while the corresponding adversarial prompt generated by GhostPrompt successfully bypasses text filters and retains the NSFW semantics. On the right, We append an instruction (shown in red text) to the prompt that guide the T2I model to generate a logo in the image, enabling the image to bypass image filters.
• Figure 4: Average time to generate successful adversarial prompts across filters. GhostPrompt (Ours) is significantly faster.
• Figure 5: CLIP Score vs. Text Bypass Rate. Dot size indicates filter scale. GhostPrompt achieves strong alignment and high bypass.