Adversarial Attacks on Parts of Speech: An Empirical Study in Text-to-Image Generation

TL;DR

The paper addresses the robustness of text-to-image diffusion models to adversarial prompts across six POS categories, expanding beyond nouns to include verbs, adjectives, adverbs, numerals, and proper nouns. It presents a gradient-based attack that perturbs prompts via adversarial suffixes, supported by a custom MS-COCO-based dataset and thorough evaluation (ASR, SemSR, human judgments). Key findings show nouns, proper nouns, and adjectives are more susceptible, while verbs, adverbs, and numerals demonstrate resilience; the mechanism hinges on the number of critical tokens and content fusion, with suffix transferability contributing to cross-prompt generalization. The work contributes a public implementation, insights into attack dynamics, and considerations for bolstering robustness in T2I systems, with practical implications for model defense and prompt engineering.

Abstract

Recent studies show that text-to-image (T2I) models are vulnerable to adversarial attacks, especially with noun perturbations in text prompts. In this study, we investigate the impact of adversarial attacks on different POS tags within text prompts on the images generated by T2I models. We create a high-quality dataset for realistic POS tag token swapping and perform gradient-based attacks to find adversarial suffixes that mislead T2I models into generating images with altered tokens. Our empirical results show that the attack success rate (ASR) varies significantly among different POS tag categories, with nouns, proper nouns, and adjectives being the easiest to attack. We explore the mechanism behind the steering effect of adversarial suffixes, finding that the number of critical tokens and content fusion vary among POS tags, while features like suffix transferability are consistent across categories. We have made our implementation publicly available at - https://github.com/shahariar-shibli/Adversarial-Attack-on-POS-Tags.

Figures (13)

• Figure 1: Examples of successful adversarial attacks on Stable Diffusion covering different POS tags drawn from our dataset. The POS tokens targeted by adversarial suffixes are highlighted in red. In addition, we observe that the attack success rate (ASR) varies significantly across POS tag categories, with features like the number of critical tokens (defined in § \ref{['sec:analysis']}, non-critical tokens are highlighted in orange) being highly associated with ASR.
• Figure 2: Average length of critical tokens across different POS tags in unrestricted and restricted settings. Exponential trend lines are included for both settings to highlight the general pattern.
• Figure 3: Schematic view of the POS-Attack pipeline. At first, hidden state representations from the CLIP text encoder using input and target token embeddings are extracted. Then, we compute loss, take gradients, and select the top-k candidate tokens for substitution. Next, we create several candidate prompts by randomly replacing multiple tokens from the pool. The candidate prompt maximizing a score function is chosen for the next optimization step.
• Figure 4: Examples of vulnerabilities revealed by SD model with prompts containing adverbs and proper nouns.
• Figure 5: Examples of vulnerabilities revealed by SD model with numerals. Each row contains 10 images with numerals one to ten from left to right sequentially. The first row contains ten images of the prompt "___ bears lying in the field" where ___ is replaced by "one" to "ten" serially. Similarly, the second, third, and fourth rows contain prompts "___ birds looking around while on the ground", "___ cows eating grass" and "___ sheep roaming in the field".