Realistic Face Reconstruction from Facial Embeddings via Diffusion Models

TL;DR

This work introduces FEM, a framework that maps face embeddings from FR or PPFR systems into the embedding space of a pre-trained, identity-preserving diffusion model (IPA-FaceID) to reconstruct realistic face images. By leveraging a Kolmogorov-Arnold-inspired embedding mapper (FEM-MLP, FEM-KAN), it enables effective embedding-to-face attacks and exposes privacy leakage risks across standard FR and privacy-protected systems. Comprehensive experiments demonstrate that FEM outperforms state-of-the-art baselines, remains robust to makeup, partial leakage, and embedding protections, and offers substantial improvements in training efficiency and inference speed. The results highlight practical privacy concerns and provide a usable tool for evaluating the safety of FR/PPFR deployments in real-world settings.

Abstract

With the advancement of face recognition (FR) systems, privacy-preserving face recognition (PPFR) systems have gained popularity for their accurate recognition, enhanced facial privacy protection, and robustness to various attacks. However, there are limited studies to further verify privacy risks by reconstructing realistic high-resolution face images from embeddings of these systems, especially for PPFR. In this work, we propose the face embedding mapping (FEM), a general framework that explores Kolmogorov-Arnold Network (KAN) for conducting the embedding-to-face attack by leveraging pre-trained Identity-Preserving diffusion model against state-of-the-art (SOTA) FR and PPFR systems. Based on extensive experiments, we verify that reconstructed faces can be used for accessing other real-word FR systems. Besides, the proposed method shows the robustness in reconstructing faces from the partial and protected face embeddings. Moreover, FEM can be utilized as a tool for evaluating safety of FR and PPFR systems in terms of privacy leakage. All images used in this work are from public datasets.

Figures (7)

• Figure 1: Sample face images from the CelebA-HQ dataset (first row) and their corresponding reconstructed face images from face embeddings of PPFR model DCTDP. The orange color value indicates confidence score (higher is better) given by commercial API Face++.
• Figure 2: Pipeline of face embedding mapping (FEM). In training, FEM learns to map between target model and the default FR of IPA-FaceID. During inference stage, trained FEM can directly reconstruct complete face images from the leaked embedding.
• Figure 3: Face embedding distributions before and after mapped by FEMs. Visualized by UMAP mcinnes2018umap.
• Figure 4: Two variants of FEM models and the process of embedding-to-embedding mapping. (a) FEM-MLP has fixed activation function. (b) FEM-KAN has learnable activation function at edges to achieve accurate non-linear mapping. (c) The direction of embedding mapping optimized by distance towards to "ground truth" face embedding $e_i$.
• Figure 5: Visual Comparison of reconstructed faces from CelebA-HQ dataset.