Adversarial Attacks on Both Face Recognition and Face Anti-spoofing Models

TL;DR

This work addresses the practical vulnerability of face recognition systems when integrated with face anti-spoofing by proposing a joint adversarial attack framework, Reference-free Multi-level Alignment (RMA). RMA combines three modules—Adaptive Gradient Maintenance (AGM) to balance FR and FAS gradients, Reference-free Intermediate Biasing (RIB) to improve FAS transferability without fixed live references, and Multi-level Feature Alignment (MFA) to enhance FR transferability across multiple intermediate layers. By formulating a dual-objective attack and enabling surrogate-model guidance, RMA achieves state-of-the-art black-box attack performance against both FR and FAS components, and across adversarially trained defenses. The approach demonstrates significant improvements in transferability and practicality for attacks on integrated FR systems, with broad implications for evaluating and improving the robustness of real-world biometric authentication pipelines.

Abstract

Adversarial attacks on Face Recognition (FR) systems have demonstrated significant effectiveness against standalone FR models. However, their practicality diminishes in complete FR systems that incorporate Face Anti-Spoofing (FAS) models, as these models can detect and mitigate a substantial number of adversarial examples. To address this critical yet under-explored challenge, we introduce a novel attack setting that targets both FR and FAS models simultaneously, thereby enhancing the practicability of adversarial attacks on integrated FR systems. Specifically, we propose a new attack method, termed Reference-free Multi-level Alignment (RMA), designed to improve the capacity of black-box attacks on both FR and FAS models. The RMA framework is built upon three key components. Firstly, we propose an Adaptive Gradient Maintenance module to address the imbalances in gradient contributions between FR and FAS models. Secondly, we develop a Reference-free Intermediate Biasing module to improve the transferability of adversarial examples against FAS models. In addition, we introduce a Multi-level Feature Alignment module to reduce feature discrepancies at various levels of representation. Extensive experiments showcase the superiority of our proposed attack method to state-of-the-art adversarial attacks.

Figures (4)

• Figure 1: Left: A comparison between previous methods and our proposed method. Adversarial examples generated by previous methods for FR systems often contain spoofing artifacts, making them easily detectable and filtered out by FAS models. In contrast, our method attacks both FR and FAS models simultaneously, improving the practicality of adversarial examples in FR systems. Right: Performance comparison based on the metrics of LPIPS, and Attack Success Rate (ASR) for FR (ASR$’$), FAS (ASR$^{*}$), and both models (ASR$^{\jmath}$).
• Figure 2: Overview of our Reference-free Multi-level Alignment (RMA) framework. (a) The Reference-free Intermediate Biasing (RIB) module biases adversarial examples toward the live image space using the intermediate loss function without overfitting to specific reference live images by a surrogate model accessible to the attacker, enhancing the attack effectiveness on black-box FAS models where direct access is restricted. (b) The Multi-level Feature Alignment (MFA) module aligns the features of adversarial examples with those of target images across multiple intermediate layers, thereby improving their transferability when attacking FR models. (c) The Adaptive Gradient Maintenance (AGM) module balances the gradients between FR and FAS by adaptively scaling their respective losses, thereby mitigating the disparities between the gradients the during each iteration.
• Figure 3: The illustration of the crafted adversarial examples.
• Figure 4: Ablation studies on the RIB module: (a) Violin plot illustrating the black-box live scores. (b) ASR$^*$ (%) results.