KAN See Your Face

TL;DR

This work introduces the first approach to exploit Kolmogorov-Arnold Network (KAN) for conducting embedding-to-face attacks against state-of-the-art (SOTA) FR and PPFR systems, and demonstrates the effectiveness of FEMs in accurate embedding mapping and face reconstruction.

Abstract

With the advancement of face reconstruction (FR) systems, privacy-preserving face recognition (PPFR) has gained popularity for its secure face recognition, enhanced facial privacy protection, and robustness to various attacks. Besides, specific models and algorithms are proposed for face embedding protection by mapping embeddings to a secure space. However, there is a lack of studies on investigating and evaluating the possibility of extracting face images from embeddings of those systems, especially for PPFR. In this work, we introduce the first approach to exploit Kolmogorov-Arnold Network (KAN) for conducting embedding-to-face attacks against state-of-the-art (SOTA) FR and PPFR systems. Face embedding mapping (FEM) models are proposed to learn the distribution mapping relation between the embeddings from the initial domain and target domain. In comparison with Multi-Layer Perceptrons (MLP), we provide two variants, FEM-KAN and FEM-MLP, for efficient non-linear embedding-to-embedding mapping in order to reconstruct realistic face images from the corresponding face embedding. To verify our methods, we conduct extensive experiments with various PPFR and FR models. We also measure reconstructed face images with different metrics to evaluate the image quality. Through comprehensive experiments, we demonstrate the effectiveness of FEMs in accurate embedding mapping and face reconstruction.

Figures (8)

• Figure 1: Sample face images from the CelebA-HQ dataset (first row) and their corresponding reconstructed face images from face templates of PPFR model DCTDP. The orange color value indicates confidence score (higher is better) given by commercial API Face++.
• Figure 2: Pipeline of face reconstruction by face embedding mapping.
• Figure 3: Two variants of FEM models and the process of embedding-to-embedding mapping. (a) FEM-MLP has fixed activation function. (b) FEM-KAN has learnable activation function at edges to achieve accurate non-linear mapping. (c) The direction of embedding mapping optimized by distance towards to 'ground truth' face embedding $e_i$.
• Figure 4: Cosine similarity distributions between input and generated faces from FEMs. ArcFace is used as target model to extract embeddings from Synth-500. FEMs are trained on FFHQ dataset.
• Figure 5: Reconstructed faces by FEM-KAN from different percentage of embedding leakage. IRSE50 is target model.