G2Face: High-Fidelity Reversible Face Anonymization via Generative and Geometric Priors

TL;DR

G2Face addresses the challenge of reversible face anonymization by preserving facial geometry and ID-irrelevant attributes while enabling accurate recovery. It integrates a Geometry-aware Identity Extraction module with a Dual Prior-guided Identity Manipulation framework and a Password Extraction module, leveraging 3DMM geometry and StyleGAN2 priors to produce realistic anonymized faces and exact recoveries. The method introduces an identity-aware feature fusion mechanism and a password-embedding scheme, achieving superior anonymization quality, recovery fidelity, and downstream data utility across extensive evaluations. This approach offers a practical, high-utility solution for privacy-preserving face data in real-world tasks and datasets.

Abstract

Reversible face anonymization, unlike traditional face pixelization, seeks to replace sensitive identity information in facial images with synthesized alternatives, preserving privacy without sacrificing image clarity. Traditional methods, such as encoder-decoder networks, often result in significant loss of facial details due to their limited learning capacity. Additionally, relying on latent manipulation in pre-trained GANs can lead to changes in ID-irrelevant attributes, adversely affecting data utility due to GAN inversion inaccuracies. This paper introduces G\textsuperscript{2}Face, which leverages both generative and geometric priors to enhance identity manipulation, achieving high-quality reversible face anonymization without compromising data utility. We utilize a 3D face model to extract geometric information from the input face, integrating it with a pre-trained GAN-based decoder. This synergy of generative and geometric priors allows the decoder to produce realistic anonymized faces with consistent geometry. Moreover, multi-scale facial features are extracted from the original face and combined with the decoder using our novel identity-aware feature fusion blocks (IFF). This integration enables precise blending of the generated facial patterns with the original ID-irrelevant features, resulting in accurate identity manipulation. Extensive experiments demonstrate that our method outperforms existing state-of-the-art techniques in face anonymization and recovery, while preserving high data utility. Code is available at https://github.com/Harxis/G2Face.

Figures (9)

• Figure 1: Current methods for face anonymization either employ a basic encoder-decoder model for identity modification (a) or modify the latent code using GAN inversion (b), both approaches often resulting in significant loss of facial detail or undesirable changes in ID-irrelevant attributes. Our proposed method intricately combines generative and geometric priors, enhanced by identity guidance, to refine the process of identity modification. This approach enables not only realistic face anonymization but also faithful recovery, all while effectively preserving the utility of the data (c).
• Figure 2: Overview of the two-stage training pipeline for our proposed G2Face. As the two stages share the same backbone, we showcase the main framework for simplicity, and a more detailed pipeline of each stage are illustrated in Fig. \ref{['fig:reversible']}. Particularly, in the anonymization stage (blue arrow), the GIE first extracts the geometry coefficients of the input face $X$ and combines them with a randomly generated dummy ID embedding $e'_{id}$ to derive the geometry-aware identity embedding. The DPIM then generates ID-relevant features based on this embedding. These features are adaptively blended with ID-irrelevant features from the original image using the proposed identity-aware feature fusion block (IFF) in a multi-scale manner, resulting in the final anonymized face image $Y$. In the recovery stage (orange arrow), the recovered image $\hat{X}$ is generated using the geometry-aware identity embedding from the real ID embedding $e_{id}$ and the geometry coefficients of the anonymized face $Y$, along with the multi-scale feature of $Y$ by the same process as the anonymous stage. Note that during each single stage, only the corresponding ID embedding is used as the input.
• Figure 3: Visualization of adaptive mask $M$, ID-irrelevant features $F_{face}$, and generative features $F_{inter}$. $M$ serves as a key indicator to guide the integration of $F_{face}$ and $F_{inter}$.
• Figure 4: The inference pipeline for our proposed reversible face anonymization. (a) Face anonymization. (b) Face recovery.
• Figure 5: Qualitative comparison of face anonymization and recovery among different methods on the CelebA-HQ dataset karras2018progressive. (a)-(f) are the original face images, the anonymization results of CIAGAN maximov2020ciagan, FIT gu2020password, RiDDLE li2023riddle, FALCO barattin2023attribute, and our method, respectively. (g) to (i) are the recovery results of FIT gu2020password, RiDDLE li2023riddle, and our G2Face, respectively. Note that only FIT gu2020password and RiDDLE li2023riddle support reversible face anonymization.