Optimal-Landmark-Guided Image Blending for Face Morphing Attacks

TL;DR

This work targets face morphing attacks on automatic face recognition systems by marrying optimized landmark manipulation with graph-based appearance warping. It introduces two modules: landmark blending to produce an optimal morphed landmark set $L_m$, and landmark guided image blending that uses Graph Convolutional Networks on a fully connected bipartite landmark graph to iteratively fuse appearance and shape features. The method employs adversarial training with two discriminators and a multi-component loss to produce high-quality morphed images while preserving identity cues from both contributing subjects. Experimental results on FERET/FRGC datasets show superior morph realism and higher vulnerability to multiple FRSs (as measured by MMPMR) compared with state-of-the-art landmark- and generation-based approaches, alongside favorable perceptual metrics (PSNR/SSIM). This work highlights a significant threat to FRS security and underscores the need for robust morphing attack detection and defense strategies.

Abstract

In this paper, we propose a novel approach for conducting face morphing attacks, which utilizes optimal-landmark-guided image blending. Current face morphing attacks can be categorized into landmark-based and generation-based approaches. Landmark-based methods use geometric transformations to warp facial regions according to averaged landmarks but often produce morphed images with poor visual quality. Generation-based methods, which employ generation models to blend multiple face images, can achieve better visual quality but are often unsuccessful in generating morphed images that can effectively evade state-of-the-art face recognition systems~(FRSs). Our proposed method overcomes the limitations of previous approaches by optimizing the morphing landmarks and using Graph Convolutional Networks (GCNs) to combine landmark and appearance features. We model facial landmarks as nodes in a bipartite graph that is fully connected and utilize GCNs to simulate their spatial and structural relationships. The aim is to capture variations in facial shape and enable accurate manipulation of facial appearance features during the warping process, resulting in morphed facial images that are highly realistic and visually faithful. Experiments on two public datasets prove that our method inherits the advantages of previous landmark-based and generation-based methods and generates morphed images with higher quality, posing a more significant threat to state-of-the-art FRSs.

Figures (4)

• Figure 1: Compared to existing landmark-based methods, our proposed approach can generate high-quality morphs while also increasing the vulnerability of facial recognition systems. Traditional landmark-based methods, as depicted in the red box, partition the face into multiple triangles, resulting in discontinuities and unnatural phenomena at the connections between triangles in fine-detail areas (red circles 2, 3). Additionally, inaccurate landmark detection or missing landmarks can lead to duplicated images in the eye region (red circle 1), and jagged edges can appear along the edges of the face (red circle 4). In contrast, our landmark blending module enhances the similarity between the morphed and original images by optimizing the morphed landmarks. Furthermore, we use GCNs to analyze the relationships between landmarks, minimizing facial artifacts in the generated morphed images. (Arcface DBLP:conf/cvpr/DengGXZ19 FRS computes the similarity score to measure how similar a morphed image is to the two original images it was created from.)
• Figure 2: Workflow of Optimal Landmark-Guided Morph Generation. Our proposed method consists of two modules: landmark blending and landmark guided image blending. The former is designed to identify optimized morphed landmark $L_m$ that preserves facial structures similar to both contributing images. The morphed landmarks guide the blending process of facial appearances in the landmark guided image blending module. To accomplish this, we employ GCNs to naturally warp the original image by inferring the relationships between the morphed landmarks $L_m$ and the contributing original landmarks $L_1$ and $L_2$. Following $N$ iterations of GCN inference, the intermediate warping images $I^{'}_{m_1}$ and $I^{'}_{m_2}$ are combined to produce the final morphed image.
• Figure 3: Analysis of DET curves for HOG+SVM and MixfaceNets across diverse test sets and varied training data configurations.
• Figure 4: Comparing the Quality of Face Morphs Generated by the Proposed Method and Existing Approaches: A Qualitative Analysis.