Rethinking the Vulnerabilities of Face Recognition Systems:From a Practical Perspective

TL;DR

This work reveals practical vulnerabilities in face recognition systems by introducing FIBA, an enrollment-stage facial identity backdoor that allows attackers with a wearable trigger to bypass FRS after a compromised insider enrolls a disguised identity. By optimizing a universal backdoor patch across multiple feature extractors and incorporating physical-world transformations, FIBA achieves high attack success rates across digital and real-world environments, including IoT devices with liveness verification. The study combines a user-driven motivation with extensive experiments on six models, five commercial APIs, and three edge devices, demonstrating strong generalization and transferability even under black-box conditions. To mitigate these risks, the authors propose stage-specific defenses, including data quality assurance, live detection, patch-detection methods, and adversarial training, while acknowledging limitations such as reliance on 2D printing and transferability bounds. Overall, the paper highlights a shift in the threat landscape from training-data poisoning to enrollment-stage exploitation, underscoring the need for robust, verifiable, and trusted FRS in security-critical applications.

Abstract

Face Recognition Systems (FRS) have increasingly integrated into critical applications, including surveillance and user authentication, highlighting their pivotal role in modern security systems. Recent studies have revealed vulnerabilities in FRS to adversarial (e.g., adversarial patch attacks) and backdoor attacks (e.g., training data poisoning), raising significant concerns about their reliability and trustworthiness. Previous studies primarily focus on traditional adversarial or backdoor attacks, overlooking the resource-intensive or privileged-manipulation nature of such threats, thus limiting their practical generalization, stealthiness, universality and robustness. Correspondingly, in this paper, we delve into the inherent vulnerabilities in FRS through user studies and preliminary explorations. By exploiting these vulnerabilities, we identify a novel attack, facial identity backdoor attack dubbed FIBA, which unveils a potentially more devastating threat against FRS:an enrollment-stage backdoor attack. FIBA circumvents the limitations of traditional attacks, enabling broad-scale disruption by allowing any attacker donning a specific trigger to bypass these systems. This implies that after a single, poisoned example is inserted into the database, the corresponding trigger becomes a universal key for any attackers to spoof the FRS. This strategy essentially challenges the conventional attacks by initiating at the enrollment stage, dramatically transforming the threat landscape by poisoning the feature database rather than the training data.

Figures (21)

• Figure 1: Illustration of FIBA. An insider wearing a trigger registers his/her face into the database. During authentication, attackers with the same trigger can bypass FRS.
• Figure 1: All mask types used in our experiments.
• Figure 2: Workflow of the traditional FRS.
• Figure 2: The distribution of the surveyed users.
• Figure 3: Results of user study. Question 1: Have you ever been mistaken for someone else when using FRS?Question 2: Have you ever been unrecognized when using FRS?