Face Reconstruction Transfer Attack as Out-of-Distribution Generalization

TL;DR

This work addresses the vulnerability of face recognition systems to transfer attacks by formalizing Face Reconstruction Transfer Attacks (FRTA) as an Out-Of-Distribution (OOD) generalization problem. It introduces Averaged Latent Search with Unsupervised Validation using a pseudo target (ALSUV), which optimizes multiple latents of a StyleGAN2 generator, averages latent trajectories, and uses a pseudo-target validation encoder to improve generalization to unseen encoders. Extensive experiments on LFW, CFP-FP, and AgeDB-30 across six encoders demonstrate state-of-the-art transfer attack performance, with robust transfer to unseen systems and analysis linking flat minima to better generalization. The findings highlight security risks in FRTA and provide methodological insights for evaluating and mitigating cross-encoder transfer vulnerabilities, with code to be released for replication.

Abstract

Understanding the vulnerability of face recognition systems to malicious attacks is of critical importance. Previous works have focused on reconstructing face images that can penetrate a targeted verification system. Even in the white-box scenario, however, naively reconstructed images misrepresent the identity information, hence the attacks are easily neutralized once the face system is updated or changed. In this paper, we aim to reconstruct face images which are capable of transferring face attacks on unseen encoders. We term this problem as Face Reconstruction Transfer Attack (FRTA) and show that it can be formulated as an out-of-distribution (OOD) generalization problem. Inspired by its OOD nature, we propose to solve FRTA by Averaged Latent Search and Unsupervised Validation with pseudo target (ALSUV). To strengthen the reconstruction attack on OOD unseen encoders, ALSUV reconstructs the face by searching the latent of amortized generator StyleGAN2 through multiple latent optimization, latent optimization trajectory averaging, and unsupervised validation with a pseudo target. We demonstrate the efficacy and generalization of our method on widely used face datasets, accompanying it with extensive ablation studies and visually, qualitatively, and quantitatively analyses. The source code will be released.

Figures (7)

• Figure 1: (a) Poorly generalized attack images often get rejected on unseen encoders (middle) while generalized images can bypass other unseen systems (right) like real face images (left).(b) Cosine similarity when only one latent is optimized (red) and the top 1 of multiple latent optimized (blue). Red histogram suffers with underfitting. (c) SAR of top 1 to 5 candidates of target encoder (coral bar) and our method (turquoise bar) tested on unseen encoders. Our method shows consistent correlation between rank and performance as well as better results.
• Figure 2: Overview of our method. Latents of a pre-trained generative model $G$ is optimized to ensure high similarity between the feature embedding of reconstructed samples and real feature embeddings.
• Figure 3: Results of variating hyperparameters (a) number of latents $n$, (b) size of latent averaging $t$, and (c) number of samples $k_{top}$ for unsupervised validation with LFW dataset.
• Figure 4: Loss surface landscape and contour line as heat map(a) of a single sample seen encoder with and without latent averaging applied. (b) Quantitative statistics are shown as box plot showing the first eigenvalue of hessian(left), trace of hessian(middle) for seen encoder and loss for unseen encoders measured with LFW samples.
• Figure 5: Cosine distance statistics between seen encoder's top 1, unsupervised validation's top 1, and pseudo target against real image's feature in validation feature space (a) and unseen encoder space (b) (pseudo target excluded since it only presents in validation space). (c) shows the correlation between the validation encoder's performance and performance improvement. 88

Theorems & Definitions (1)

• proof