Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Now You See Me, Now You Don't: A Unified Framework for Expression Consistent Anonymization in Talking Head Videos

Anil Egin, Andrea Tangherloni, Antitza Dantcheva

TL;DR

AnonNET addresses privacy-preserving video anonymization by obfuscating identity while preserving key facial attributes and dynamics. It introduces a diffusion-based inpainting module conditioned on demographic attributes and guided by structural priors, integrated with landmark-free motion transfer and scene-aware video reassembly. The framework achieves strong identity obfuscation with superior perceptual quality and temporal coherence across challenging datasets, outperforming several baselines in both image and video domains. These capabilities enable safer data sharing for downstream tasks such as expression analysis, tracking, and affective computing in privacy-critical contexts.

Abstract

Face video anonymization is aimed at privacy preservation while allowing for the analysis of videos in a number of computer vision downstream tasks such as expression recognition, people tracking, and action recognition. We propose here a novel unified framework referred to as Anon-NET, streamlined to de-identify facial videos, while preserving age, gender, race, pose, and expression of the original video. Specifically, we inpaint faces by a diffusion-based generative model guided by high-level attribute recognition and motion-aware expression transfer. We then animate deidentified faces by video-driven animation, which accepts the de-identified face and the original video as input. Extensive experiments on the datasets VoxCeleb2, CelebV-HQ, and HDTF, which include diverse facial dynamics, demonstrate the effectiveness of AnonNET in obfuscating identity while retaining visual realism and temporal consistency. The code of AnonNet will be publicly released.

Now You See Me, Now You Don't: A Unified Framework for Expression Consistent Anonymization in Talking Head Videos

TL;DR

AnonNET addresses privacy-preserving video anonymization by obfuscating identity while preserving key facial attributes and dynamics. It introduces a diffusion-based inpainting module conditioned on demographic attributes and guided by structural priors, integrated with landmark-free motion transfer and scene-aware video reassembly. The framework achieves strong identity obfuscation with superior perceptual quality and temporal coherence across challenging datasets, outperforming several baselines in both image and video domains. These capabilities enable safer data sharing for downstream tasks such as expression analysis, tracking, and affective computing in privacy-critical contexts.

Abstract

Face video anonymization is aimed at privacy preservation while allowing for the analysis of videos in a number of computer vision downstream tasks such as expression recognition, people tracking, and action recognition. We propose here a novel unified framework referred to as Anon-NET, streamlined to de-identify facial videos, while preserving age, gender, race, pose, and expression of the original video. Specifically, we inpaint faces by a diffusion-based generative model guided by high-level attribute recognition and motion-aware expression transfer. We then animate deidentified faces by video-driven animation, which accepts the de-identified face and the original video as input. Extensive experiments on the datasets VoxCeleb2, CelebV-HQ, and HDTF, which include diverse facial dynamics, demonstrate the effectiveness of AnonNET in obfuscating identity while retaining visual realism and temporal consistency. The code of AnonNet will be publicly released.
Paper Structure (31 sections, 7 figures, 5 tables)

This paper contains 31 sections, 7 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Image Anonymization
  4. Motion Transfer
  5. Video Anonymization
  6. Method
  7. Overview of the Proposed Framework
  8. Video Preprocessing and Scene Detection
  9. (1) Scene Change Detection.
  10. (2) Frontal Selection
  11. Face Detection and Attribute-Guided Prompt Generation
  12. (3) Attribute Recognition.
  13. Diffusion-Based Inpainting
  14. (4) Diffusion-Based Inpainting.
  15. Anonymity Verification and Motion Transfer
  16. ...and 16 more sections

Figures (7)

  • Figure 1: Qualitative comparison between original and anonymized face pairs. Left: CelebA-HQ. Right: LFW. Each row shows original/anonymized pairs.
  • Figure 2: Overview of our multi-stage anonymization AnonNET-pipeline. (1) Scene changes are detected, and identities are tracked. (2) Faces are detected and single frontal frame is selected per scene identity. (3) Facial attributes are recognized. (4) A diffusion-based model inpaints the masked face. (5) Current anonymity is evaluated. (6) Landmark-free motion transfer reintroduces natural head movement. (7) Frames are reassembled for a coherent output video.
  • Figure 3: Overview of the expression-consistent face anonymization module in AnonNET. (1) A face detection and frontal selection stage extracts a frame containing a frontal face from the input scene video. Then this frontal image is processed in parallel by two branches: (2) An attribute recognition module that infers semantic attributes such as age, gender, race, and expression; (3) a ControlNet module, which extracts structural guidance (e.g., face mask, lineart, pose) for conditioning the generative model. (4) Stable Diffusion based on U-Net synthesizes an anonymized face conditioned on both, extracted attributes and ControlNet features.
  • Figure 4: Qualitative face anonymization results pertained to the CelebA-HQ dataset. Each row corresponds to an input image (left column), and columns show outputs from various image-anonymization methods.
  • Figure 5: Qualitative comparison of original and anonymized frames pertained to the VoxCeleb2 dataset using AnonNET. Each column shows original/anonymized pairs.
  • ...and 2 more figures