Unveiling the Attribute Misbinding Threat in Identity-Preserving Models

TL;DR

The paper identifies a safety vulnerability in identity-preserving diffusion models where attributes can be misbound to the target identity, enabling NSFW outputs even when prompts appear benign. It introduces the Misbinding Prompt evaluation set and the ABSS metric to jointly assess content fidelity and safety, backed by a formal framework of misbinding strategies and risk scoring. Empirical results show that Misbinding Prompts outperform existing baselines in bypassing safety filters and in inducing NSFW outputs, particularly for identity-preserving models, underscoring the need for defenses beyond prompt filtering. These contributions provide a concrete, testable paradigm for evaluating and benchmarking safety risks in identity-preserving generation while highlighting gaps in current safeguards.

Abstract

Identity-preserving models have led to notable progress in generating personalized content. Unfortunately, such models also exacerbate risks when misused, for instance, by generating threatening content targeting specific individuals. This paper introduces the \textbf{Attribute Misbinding Attack}, a novel method that poses a threat to identity-preserving models by inducing them to produce Not-Safe-For-Work (NSFW) content. The attack's core idea involves crafting benign-looking textual prompts to circumvent text-filter safeguards and leverage a key model vulnerability: flawed attribute binding that stems from its internal attention bias. This results in misattributing harmful descriptions to a target identity and generating NSFW outputs. To facilitate the study of this attack, we present the \textbf{Misbinding Prompt} evaluation set, which examines the content generation risks of current state-of-the-art identity-preserving models across four risk dimensions: pornography, violence, discrimination, and illegality. Additionally, we introduce the \textbf{Attribute Binding Safety Score (ABSS)}, a metric for concurrently assessing both content fidelity and safety compliance. Experimental results show that our Misbinding Prompt evaluation set achieves a \textbf{5.28}\% higher success rate in bypassing five leading text filters (including GPT-4o) compared to existing main-stream evaluation sets, while also demonstrating the highest proportion of NSFW content generation. The proposed ABSS metric enables a more comprehensive evaluation of identity-preserving models by concurrently assessing both content fidelity and safety compliance.

Figures (7)

• Figure 1: Demonstration of the proposed Attribute Misbinding Attack against five leading identity-preserving models. To avoid infringing upon the portrait rights of real individuals, all reference face images used in this demonstration are portraits generated by StyleGAN2.
• Figure 2: The proposed framework for generating Misbinding Prompt evaluation set and evaluating the safety of identity-preserving models. The framework consists of four stages: (1) Sensitive Term Expansion, to methodically broaden the vocabulary of sensitive terms; (2) Attribute Misbinding Attack, to programmatically create prompts via predefined strategies; (3) Diffusion Generation, to use prompts and identity reference images for synthesis; (4) Attribute Binding Safety Score Calculation, where an MLLM assesses the output to calculate the final score.
• Figure 3: Illustration of bypassing safety filters via Attribute Misbinding.
• Figure 4: The system prompt used to instruct a Large Language Model (LLM) to act as an automated text filter. The prompt defines the scoring task (0-100 scale), the risk categories for evaluation, and the required JSON output format for the safety score.
• Figure 5: The system prompt used to instruct a Multimodal Large Language Model (MLLM) to act as an automated safety evaluator. The prompt defines the scoring task (0-100 scale), the risk categories to consider, and the required JSON output format for the final safety score.