Unified Prompt Attack Against Text-to-Image Generation Models

TL;DR

This work addresses the security of text-to-image generation by proposing UPAM, a unified gradient-based attack that simultaneously bypasses textual filters and visual checkers in black-box T2I APIs. It introduces Sphere-Probing Learning to estimate gradients without image outputs, Semantic-Enhancing Learning to align generated content with attacker intent, In-context Naturalness Enhancement for natural prompts, and Transferable Attack Learning to enable few-shot queries. The framework achieves strong attack effectiveness, efficiency, and naturalness, with ablations and protocol variants demonstrating the value of each component and transferability across models. The results highlight both the potential vulnerabilities of current T2I services and the need for robust defenses, while offering a framework that can inform defense improvements and serve as a security-vulnerability detector for API providers.

Abstract

Text-to-Image (T2I) models have advanced significantly, but their growing popularity raises security concerns due to their potential to generate harmful images. To address these issues, we propose UPAM, a novel framework to evaluate the robustness of T2I models from an attack perspective. Unlike prior methods that focus solely on textual defenses, UPAM unifies the attack on both textual and visual defenses. Additionally, it enables gradient-based optimization, overcoming reliance on enumeration for improved efficiency and effectiveness. To handle cases where T2I models block image outputs due to defenses, we introduce Sphere-Probing Learning (SPL) to enable optimization even without image results. Following SPL, our model bypasses defenses, inducing the generation of harmful content. To ensure semantic alignment with attacker intent, we propose Semantic-Enhancing Learning (SEL) for precise semantic control. UPAM also prioritizes the naturalness of adversarial prompts using In-context Naturalness Enhancement (INE), making them harder for human examiners to detect. Additionally, we address the issue of iterative queries--common in prior methods and easily detectable by API defenders--by introducing Transferable Attack Learning (TAL), allowing effective attacks with minimal queries. Extensive experiments validate UPAM's superiority in effectiveness, efficiency, naturalness, and low query detection rates.

Figures (8)

• Figure 1: T2I APIs could incorporate both textual filters and visual checkers for dual defense, which can deny the forward propagation of data when harmful information is detected schramowski2023saferombach2022high.
• Figure 2: Intuitive illustration of our SPL scheme.
• Figure 3: Overview of our UPAM framework. Initially, due to the lack of training, our adversarial prompt $\mathbf{T}^{\ast}$ is unable to bypass the API's defenses (i.e., return no image). At the pre-training phase, we propose an SPL scheme that estimates the gradient $\pmb{g}_\mathrm{spl}$ under the challenging no-result scenario, finally compelling the black-box T2I model to return images. At the fine-tuning phase, we utilize an SEL scheme to provide the gradient $\pmb{g}_\mathrm{sel}$ to align the returned images $\mathbf{I}^{\ast}$ with the target semantics.
• Figure 4: Illustration of our SPL scheme. In this figure, each point in the optimization space denotes a specific configuration of model parameters. (a) In the first stage of SPL, we optimize the model from "Deny" region towards the "Pass" region, thereby enabling the attack to bypass the defenses. (b) In the second stage of SPL, we start to optimize the model in the opposite direction to approach the boundary, thereby enhancing the likelihood of generating harmful images.
• Figure 5: Intuitive illustration of our SEL scheme. ① Firstly, We compute the gradient $\pmb{g}_\mathrm{im}$ for enhancing target harmful semantics. ② Then, we calculate negative SPL gradient $-\pmb{g}_\mathrm{spl}$, which points towards the boundary direction. ③ We calculate the component of the gradient $\pmb{g}_\mathrm{im}$ in the boundary direction, i.e., $\pmb{g}\prime_\mathrm{im}$. ④ To prevent from falling into the "deny" region, we let $\pmb{g}_\mathrm{im}$ subtract its component $\pmb{g}\prime_\mathrm{im}$, obtaining the final SEL gradient $\pmb{g}_\mathrm{sel}$, where $\pmb{g}_\mathrm{sel} = \pmb{g}_\mathrm{im} - \pmb{g}\prime_\mathrm{im}$.