SIDeR: Semantic Identity Decoupling for Unrestricted Face Privacy

TL;DR

SIDeR tackles the privacy challenge of protecting facial data while preserving machine-level identity verification by decoupling identity from human-visible appearance in a diffusion-based framework. It introduces a Semantic-Decoupled Adversarial Generation Module to produce visually anonymous yet identity-consistent faces, and a Conditionally Reversible Module that embeds the original identity into a carrier via nested invertible networks for authorized near-lossless recovery. Through momentum-based latent-space optimization, ensemble identity losses, and semantic guidance, SIDeR achieves strong black-box privacy protection and robust recovery, demonstrated on CelebA-HQ and FFHQ with high ASR and PSNR gains. The approach offers a practical privacy mechanism for biometric systems that requires both human-level anonymity and machine-level verifiability, with resilient performance against unauthorized access and across diverse recognition models.

Abstract

With the deep integration of facial recognition into online banking, identity verification, and other networked services, achieving effective decoupling of identity information from visual representations during image storage and transmission has become a critical challenge for privacy protection. To address this issue, we propose SIDeR, a Semantic decoupling-driven framework for unrestricted face privacy protection. SIDeR decomposes a facial image into a machine-recognizable identity feature vector and a visually perceptible semantic appearance component. By leveraging semantic-guided recomposition in the latent space of a diffusion model, it generates visually anonymous adversarial faces while maintaining machine-level identity consistency. The framework incorporates momentum-driven unrestricted perturbation optimization and a semantic-visual balancing factor to synthesize multiple visually diverse, highly natural adversarial samples. Furthermore, for authorized access, the protected image can be restored to its original form when the correct password is provided. Extensive experiments on the CelebA-HQ and FFHQ datasets demonstrate that SIDeR achieves a 99% attack success rate in black-box scenarios and outperforms baseline methods by 41.28% in PSNR-based restoration quality.

Figures (8)

• Figure 1: Illustration of visual-machine divergence where the machine-level identity is preserved through adversarial anchoring. The protected image maintains machine-level identity consistency with the original while supporting authorized reversible recovery. Unauthorized access disrupts the alignment between visual semantics and identity features.
• Figure 2: Achieve visual identity information hiding through deconstruction of facial identity, with the protected image maintaining identity consistency with the original at the machine recognition level, while supporting reversible recovery under legitimate authorization; if maliciously stolen by unauthorized users, the consistency of appearance semantics and visual identity will be lost.
• Figure 3: Qualitative comparison of adversarial examples on the CelebA-HQ dataset. Compared to three state-of-the-art adversarial methods, SIDeR generates examples with higher visual fidelity, particularly in facial details, background consistency, and overall naturalness.
• Figure 4: A visualization of SIDeR's multi-level recovery capabilities. The figure demonstrates a high degree of visual consistency between different stages: the top three rows represent the original carrier image "Cover," the decoy image "Decoy," and the secret image to be hidden "Secret," respectively; the "SIDeR" row corresponds to the final generated protected image; "Recover-1" and "Recover-2" represent the extracted intermediate decoy layer and the final recovered secret image, respectively. The outputs of all stages maintain natural image lighting and identity features, with no obvious visual degradation observed.
• Figure 5: Visual comparison of image recovery quality. Compared to AVIH and Diff-Privacy, our proposed SIDeR significantly reduces blurring and artifacts, maintaining superior structural fidelity and visual details on both FFHQ and CelebA-HQ datasets.