Hierarchical Generative Network for Face Morphing Attacks

TL;DR

This work tackles the challenge of producing high-quality, identity-preserving face morphs that can threaten face recognition systems. By introducing HGFM, a hierarchical GAN with one global and six local morph networks plus a fusion stage, and a mask-guided image blending module, the method captures both global structure and local facial-region details while removing artifacts outside the face. The model is trained with a multi-term loss that combines geometry, identity from multiple FR models, appearance, and adversarial objectives, and is evaluated on FERET, FRLL, and FRGC across several morphing baselines. Results show HGFM achieves strong attack performance (MMPMR) and robust MAD evasion on several datasets, with ablations highlighting the importance of local networks and the blending module for quality and identity preservation, suggesting significant practical threat potential to deployed FRSs.

Abstract

Face morphing attacks circumvent face recognition systems (FRSs) by creating a morphed image that contains multiple identities. However, existing face morphing attack methods either sacrifice image quality or compromise the identity preservation capability. Consequently, these attacks fail to bypass FRSs verification well while still managing to deceive human observers. These methods typically rely on global information from contributing images, ignoring the detailed information from effective facial regions. To address the above issues, we propose a novel morphing attack method to improve the quality of morphed images and better preserve the contributing identities. Our proposed method leverages the hierarchical generative network to capture both local detailed and global consistency information. Additionally, a mask-guided image blending module is dedicated to removing artifacts from areas outside the face to improve the image's visual quality. The proposed attack method is compared to state-of-the-art methods on three public datasets in terms of FRSs' vulnerability, attack detectability, and image quality. The results show our method's potential threat of deceiving FRSs while being capable of passing multiple morphing attack detection (MAD) scenarios.

Figures (5)

• Figure 1: Illustration of different face morphing methods. 'Sim' denotes the similarity between morphed and contributing images, which is computed by using ElasticFace elasticface.
• Figure 2: Overview of the proposed HGFM method. It consists of two modules, the hierarchical GAN for face morphing and mask-guided image blending. The former uses hierarchical generative network containing a global morph net $G_{global}$ and six local morph nets $G_{l*}$ to generate a global morphed image $I_{global}^m$ and a local morphed image $I_{local}^m$, respectively, and then fuse them to obtain an intermediate morphed image $I_M$. The latter uses a pre-trained StyleGAN2 model to generate an auxiliary morphed image $I_{aux}^m$, and the face parser is used to define the face masks $M_{m}^{face}$ and $M_{aux}^{face}$ of $I_M$ and $I_{aux}^m$ for image blending. The final result $I_M'$ is generated by combining $I_M$ and $I_{aux}^m$ according to $M_{aux}^{face}$.
• Figure 3: Examples of morphs generated by different morphing attack methods on three benchmarks. Please zoom in to see the differences among the images, especially in regions susceptible to artifacts such as the eyes, nose, mouth, and hair.
• Figure 4: Comparison of different blending schemes. From left to right: The two contributing images (BF1 and BF2), the auxiliary image generated by StyleGAN2, the morphed images generated via image blending using BF1 and BF2 simultaneously, using BF1 and BF2 sequentially, and using the StyleGAN2-generated auxiliary image. As the similarity (Sim) scores suggest, the last blending scheme achieves more balanced identity preservation between the two contributing subjects.
• Figure 5: Morphed images generated by different variants of our HGFM method in the ablation study. (a) Without geometry loss, (b) Without combined identity loss, (c) Without local morph network and local loss, (d) Without mask-guided blending, and (e) Using Poisson blending instead. Texture degradation, poor geometry and lighting preservation, ghosting, blurring, and unnatural features can be observed in these results, respectively.