Face Anonymization Made Simple

TL;DR

This work uses diffusion models with only a reconstruction loss to achieve state-of-the-art performance in three key areas: identity anonymization, facial attribute preservation, and image quality.

Abstract

Current face anonymization techniques often depend on identity loss calculated by face recognition models, which can be inaccurate and unreliable. Additionally, many methods require supplementary data such as facial landmarks and masks to guide the synthesis process. In contrast, our approach uses diffusion models with only a reconstruction loss, eliminating the need for facial landmarks or masks while still producing images with intricate, fine-grained details. We validated our results on two public benchmarks through both quantitative and qualitative evaluations. Our model achieves state-of-the-art performance in three key areas: identity anonymization, facial attribute preservation, and image quality. Beyond its primary function of anonymization, our model can also perform face swapping tasks by incorporating an additional facial image as input, demonstrating its versatility and potential for diverse applications. Our code and models are available at https://github.com/hanweikung/face_anon_simple .

Figures (21)

• Figure 1: Our face anonymization technique preserves the original facial expressions, head positioning, eye direction, and background elements, effectively masking identity while retaining other crucial details. The anonymized face blends seamlessly into its original photograph, making it ideal for diverse real-world applications.
• Figure 2: Our network leverages the face swapping mechanism for face anonymization. In both cases, the system encodes source and driving images into latent space and processes them through respective ReferenceNet models. These images are also encoded into intermediate embeddings that guide the UNet via cross-attention. The UNet incorporates states from both ReferenceNet models through concatenation, enabling the transfer of details from source and driving images through self-attention. Using these learned features and intermediate embeddings, the UNet generates the output image. For face anonymization, we use the same image as both source and driving input. However, we modify the intermediate embedding and state from the source ReferenceNet model to achieve the desired anonymization effect.
• Figure 3: For each row, we show the identity distance of each image from the original image in that row, as calculated by the ArcFace deng2019arcface recognition model. The results indicate that the recognition model can generate inaccurate identity distances. It may assign a greater identity distance to two images of the same person than to two images of different people due to variations in head pose, facial expressions, or lighting conditions.
• Figure 4: Facial images generated with different degrees of anonymization. Each generated image reflects a different degree of anonymity applied to the original face. Alongside each generated image is a cosine distance score, calculated using the FaceNet schroff2015facenet recognition model. This score quantifies how different the anonymized face is from the original in terms of identity features.
• Figure 5: Various anonymized versions created from a single original identity, each using a different integer seed value. For each anonymized version, we present the cosine distance from its original identity, calculated using the FaceNet schroff2015facenet recognition model.