MacPrompt: Maraconic-guided Jailbreak against Text-to-Image Models

TL;DR

MacPrompt is introduced, a novel black-box and cross-lingual attack that reveals previously overlooked vulnerabilities in T2I safety mechanisms and underscores the pressing need to reassess the robustness of existing T2I safety mechanisms against linguistically diverse and fine-grained adversarial strategies.

Abstract

Text-to-image (T2I) models have raised increasing safety concerns due to their capacity to generate NSFW and other banned objects. To mitigate these risks, safety filters and concept removal techniques have been introduced to block inappropriate prompts or erase sensitive concepts from the models. However, all the existing defense methods are not well prepared to handle diverse adversarial prompts. In this work, we introduce MacPrompt, a novel black-box and cross-lingual attack that reveals previously overlooked vulnerabilities in T2I safety mechanisms. Unlike existing attacks that rely on synonym substitution or prompt obfuscation, MacPrompt constructs macaronic adversarial prompts by performing cross-lingual character-level recombination of harmful terms, enabling fine-grained control over both semantics and appearance. By leveraging this design, MacPrompt crafts prompts with high semantic similarity to the original harmful inputs (up to 0.96) while bypassing major safety filters (up to 100%). More critically, it achieves attack success rates as high as 92% for sex-related content and 90% for violence, effectively breaking even state-of-the-art concept removal defenses. These results underscore the pressing need to reassess the robustness of existing T2I safety mechanisms against linguistically diverse and fine-grained adversarial strategies.

Figures (5)

• Figure 1: Cross-lingual prompts composed from other languages can trigger the same visual semantics as the original English prompt in SD v2.1.
• Figure 2: Overview of the MacPrompt framework. We assume the original prompt contains sensitive words and is blocked by existing safety filters. Here, $\beta_1, \beta_2$, and $\alpha$ are optimization parameters during macaronic substitutes construction process.
• Figure 3: Cross-lingual candidates selection pipeline. The sensitive word is translated into multiple languages, inserted into templates to generate images, and evaluated.
• Figure 4: Visualization of images generated from adversarial prompts targeting NSFW concepts and banned objects across different models; the top shows outputs for sexual and violent concepts, while the bottom shows banned objects.
• Figure 5: Visualization of semantic embeddings for banned objects and their macaronic substitutes, along with corresponding generated images.