Controllable Localized Face Anonymization Via Diffusion Inpainting
Ali Salar, Qing Liu, Guoying Zhao
TL;DR
The paper tackles privacy-preserving face anonymization while maintaining downstream utility. It introduces a diffusion-inpainting framework built on a pre-trained encoder, masked latent guidance, and an adaptive attribute-guidance module that steers reverse diffusion via gradient corrections toward a synthesized target image. This enables controllable attribute changes and localized anonymization without additional training, validated on CelebA-HQ and FFHQ where it achieves stronger anonymization (lower Re-ID) and better image quality (FID, Visual-DNA) than state-of-the-art baselines, with faster inference. The approach balances privacy and utility, though it relies on target-image guidance and could benefit from future text-based or disentangled controls to further reduce dependency on explicit targets.
Abstract
The growing use of portrait images in computer vision highlights the need to protect personal identities. At the same time, anonymized images must remain useful for downstream computer vision tasks. In this work, we propose a unified framework that leverages the inpainting ability of latent diffusion models to generate realistic anonymized images. Unlike prior approaches, we have complete control over the anonymization process by designing an adaptive attribute-guidance module that applies gradient correction during the reverse denoising process, aligning the facial attributes of the generated image with those of the synthesized target image. Our framework also supports localized anonymization, allowing users to specify which facial regions are left unchanged. Extensive experiments conducted on the public CelebA-HQ and FFHQ datasets show that our method outperforms state-of-the-art approaches while requiring no additional model training. The source code is available on our page.