IDProtector: An Adversarial Noise Encoder to Protect Against ID-Preserving Image Generation

TL;DR

IDProtector presents a fast, universal defense against encoder-based ID-preserving image generation by learning a ViT-based adversarial noise encoder that adds imperceptible perturbations to portrait photos. The method optimizes a composite loss that attacks multiple victim embeddings (ArcFace and CLIP variants) across four prominent encoder-based pipelines, while employing face-localization priors and affine-based augmentations to ensure robustness and efficient training. It achieves substantial identity-shift in generated results, outperforms baselines in ISM reductions, and remains effective on unseen data and even closed-source models, with protection times around 0.173 seconds per image and better perceptual quality than many baselines. The work advances practical portrait privacy by enabling universal, real-time protection against modern ID-preserving generative systems, while acknowledging a trade-off with Imperceptibility and outlining paths to further minimize visual artifacts.

Abstract

Recently, zero-shot methods like InstantID have revolutionized identity-preserving generation. Unlike multi-image finetuning approaches such as DreamBooth, these zero-shot methods leverage powerful facial encoders to extract identity information from a single portrait photo, enabling efficient identity-preserving generation through a single inference pass. However, this convenience introduces new threats to the facial identity protection. This paper aims to safeguard portrait photos from unauthorized encoder-based customization. We introduce IDProtector, an adversarial noise encoder that applies imperceptible adversarial noise to portrait photos in a single forward pass. Our approach offers universal protection for portraits against multiple state-of-the-art encoder-based methods, including InstantID, IP-Adapter, and PhotoMaker, while ensuring robustness to common image transformations such as JPEG compression, resizing, and affine transformations. Experiments across diverse portrait datasets and generative models reveal that IDProtector generalizes effectively to unseen data and even closed-source proprietary models.

Figures (10)

• Figure 1: Using only a single reference image, Encoder-based ID-preserving methods can generate realistic portraits, posing serious risks for malicious uses such as generation of malicious images. To prevent unauthorized ID-preserving generation, we propose IDProtector, which adds small perturbations to images. The protected images could disrupt ID-preserving generation by misleading the customization models to generate a dissimilar face, thereby achieving protection.
• Figure 2: Overall schematics of our method. Our method's key design includes the noise encoder, loss functions, and the gradient optimization path that allows backpropagation. During the protection phase, an image is first resized to 224$\times$224 and sent to an adversarial noise encoder model. The model outputs a perturbation that can be resized back and added to the image, achieving an adversarial attack effect -- the protected image prevents subsequent pipelines from correctly extracting facial feature information, thereby hindering face similarity in face generation tasks. This achieves the goal of ID protection.
• Figure 3: Qualitative comparison with baseline. Images protected by our method cannot be used to generate similar faces. Compared to baseline protection methods, under the same perturbation bound of 9/255 ($\epsilon=3.5\%$), ours causes more significant facial differences.
• Figure 4: ID protection performance on unseen generators.
• Figure 5: Robustness evaluation of IDProtector under various distortions. Despite geometric distortions like affine transformations, IDProtector effectively misleads all four models into generating dissimilar faces or even artifacts.