Enhancing Facial Privacy Protection via Weakening Diffusion Purification

TL;DR

This work tackles facial privacy protection against unauthorized AFR by exploiting latent diffusion models to edit latent codes rather than pixel space. It introduces learnable unconditional embeddings (null-text guidance) and self-attention-based structure preservation to weaken diffusion purification and preserve image quality while maintaining impersonation strength. The two-stage approach—unconditional-embedding learning followed by adversarial latent-code optimization—yields improved protection transferability across multiple FR models and datasets (CelebA-HQ, LADN) with competitive visual fidelity. The method demonstrates robustness to common countermeasures and outperforms state-of-the-art baselines on PSR and related image-quality metrics, offering a practical privacy-protection tool while highlighting ethical considerations around impersonation versus obfuscation. Overall, the paper provides a scalable, transfer-based black-box strategy for targeted de-identification via latent-diffusion editing that balances privacy efficacy and perceptual realism.

Abstract

The rapid growth of social media has led to the widespread sharing of individual portrait images, which pose serious privacy risks due to the capabilities of automatic face recognition (AFR) systems for mass surveillance. Hence, protecting facial privacy against unauthorized AFR systems is essential. Inspired by the generation capability of the emerging diffusion models, recent methods employ diffusion models to generate adversarial face images for privacy protection. However, they suffer from the diffusion purification effect, leading to a low protection success rate (PSR). In this paper, we first propose learning unconditional embeddings to increase the learning capacity for adversarial modifications and then use them to guide the modification of the adversarial latent code to weaken the diffusion purification effect. Moreover, we integrate an identity-preserving structure to maintain structural consistency between the original and generated images, allowing human observers to recognize the generated image as having the same identity as the original. Extensive experiments conducted on two public datasets, i.e., CelebA-HQ and LADN, demonstrate the superiority of our approach. The protected faces generated by our method outperform those produced by existing facial privacy protection approaches in terms of transferability and natural appearance.

Figures (11)

• Figure 1: The overview of the proposed framework for facial privacy protection. Our novel approach leverages Stable Diffusion Stable_diff_2022_CVPR to adversarially modify the latent code $z_{adv}$, enabling subtle and controlled alterations to identity-specific features, ensuring effective facial privacy protection while maintaining high visual quality. Unconditional embeddings are proposed as null-text guidance to weaken diffusion purification and enhance protection performance. Self-attention guidance is employed to preserve the structural integrity of the image, ensuring the generated faces remain visually consistent with the original while maintaining high protection efficacy.
• Figure 2: The improvements gained by incorporating learned unconditional embeddings during adversarial image generation. The first row presents original images. The second row shows protected images generated without unconditional embeddings as the null-text guidance. The third row displays protected images generated with our learned unconditional embeddings as the null-text guidance. The protection capability of the protected images is enhanced with the null-text guidance.
• Figure 3: The self-attention map Transformers_2017_ANIPS visualization shows the top seven components extracted through singular value decomposition SVD_2003_Springer. The above shows the self-attention maps corresponding to the original image, while the maps for the protected image are shown below. Despite the protection, the structural details of the original image are well preserved in the protected images.
• Figure 4: Comparison of visual quality between recent facial privacy protection methods. The absolute difference between the generated and original images is shown below each protected face.
• Figure 5: Ablation study to evaluate the contributions of different loss items. The left figure illustrates the protection success rate (PSR) under different conditions, with and without null-text and self-attention guidances, while the right one evaluates the visual quality by Fréchet inception distance (FID). The timestamp $t$ marks the point at which adversarial learning begins.