BSPA: Exploring Black-box Stealthy Prompt Attacks against Image Generators

TL;DR

This work tackles the safety vulnerabilities of large image generators by proposing BSPA, a black-box stealthy prompt attack that uses a dense text retriever and pseudo-labeling to generate diverse, covert prompts that bypass text filters and induce NSFW outputs. It converts zeroth-order black-box optimization into gradient-based search over a large sensitive-word retrieval space, enabling realistic attacks that reflect API-user behavior. The authors introduce NSFWeval, a 3,000-prompt benchmark, and a resilient text filter (RSF) to assess and mitigate such attacks across open-source and released APIs, including Stable Diffusion XL, Midjourney, and DALL-E versions. Key findings show BSPA significantly increases attack success rates while reducing detectable toxicity, highlighting the need for robust, multi-layer defense strategies and providing a framework for evaluating and improving image-generator safety in practice.

Abstract

Extremely large image generators offer significant transformative potential across diverse sectors. It allows users to design specific prompts to generate realistic images through some black-box APIs. However, some studies reveal that image generators are notably susceptible to attacks and generate Not Suitable For Work (NSFW) contents by manually designed toxin texts, especially imperceptible to human observers. We urgently need a multitude of universal and transferable prompts to improve the safety of image generators, especially black-box-released APIs. Nevertheless, they are constrained by labor-intensive design processes and heavily reliant on the quality of the given instructions. To achieve this, we introduce a black-box stealthy prompt attack (BSPA) that adopts a retriever to simulate attacks from API users. It can effectively harness filter scores to tune the retrieval space of sensitive words for matching the input prompts, thereby crafting stealthy prompts tailored for image generators. Significantly, this approach is model-agnostic and requires no internal access to the model's features, ensuring its applicability to a wide range of image generators. Building on BSPA, we have constructed an automated prompt tool and a comprehensive prompt attack dataset (NSFWeval). Extensive experiments demonstrate that BSPA effectively explores the security vulnerabilities in a variety of state-of-the-art available black-box models, including Stable Diffusion XL, Midjourney, and DALL-E 2/3. Furthermore, we develop a resilient text filter and offer targeted recommendations to ensure the security of image generators against prompt attacks in the future.

Figures (6)

• Figure 1: Schematic illustrations of the comparisons across various prompt attackers. Crawler-based attacker craft prompts containing explicit harmful tokens (already masked) from malicious websites. Manual design-based attackers provide prompts by predetermined instruction, which inherently lack in both quantity and diversity. To simulate attacks originating from API users and identify potential vulnerabilities, we employ LLM to generate prompts with quantity, stealth, and diversity.
• Figure 2: An overview of the training paradigm of black-box stealthy prompt attack (BSPA). Left: Our method transforms zeroth-order optimization into gradient-based optimization through the involvement of a retriever. This operation effectively employs text/image filter scores to tune the retrieval space of sensitive words for matching the input prompts. Right: We embed and retrieve sensitive words by a dense retrieval model, which is characterized by minimal optimization effort and expansive retrieval space.
• Figure 3: Examples of generated images by using prompts from BSPA.
• Figure 4: Ablation studies of $\mathcal{L}_{div}$.We evaluate its impact on diversity from toxic rate and the number of tokens/sensitive words.
• Figure 5: The word cloud of BSPA prompts, we set the max words as 200.