Human-Imperceptible Physical Adversarial Attack for NIR Face Recognition Models

TL;DR

This work reveals a security vulnerability in NIR face recognition by introducing a human-imperceptible physical adversarial patch made from infrared-absorbing ink. It jointly optimizes patch shape and placement under black-box constraints using differential evolution, and leverages a BRDF-based light-reflection model to align digital perturbations with physical NIR imaging. The approach achieves a high physical-domain attack success rate (82.46% on average) and outperforms the state-of-the-art AiD baseline across multiple models and datasets, while maintaining robustness to facial pose variations. The findings emphasize the urgency of developing defenses for NIR systems against stealthy, real-world physical attacks that exploit the NIR imaging process.

Abstract

Near-infrared (NIR) face recognition systems, which can operate effectively in low-light conditions or in the presence of makeup, exhibit vulnerabilities when subjected to physical adversarial attacks. To further demonstrate the potential risks in real-world applications, we design a novel, stealthy, and practical adversarial patch to attack NIR face recognition systems in a black-box setting. We achieved this by utilizing human-imperceptible infrared-absorbing ink to generate multiple patches with digitally optimized shapes and positions for infrared images. To address the optimization mismatch between digital and real-world NIR imaging, we develop a light reflection model for human skin to minimize pixel-level discrepancies by simulating NIR light reflection. Compared to state-of-the-art (SOTA) physical attacks on NIR face recognition systems, the experimental results show that our method improves the attack success rate in both digital and physical domains, particularly maintaining effectiveness across various face postures. Notably, the proposed approach outperforms SOTA methods, achieving an average attack success rate of 82.46% in the physical domain across different models, compared to 64.18% for existing methods. The artifact is available at https://anonymous.4open.science/r/Human-imperceptible-adversarial-patch-0703/.

Figures (7)

• Figure 1: The diagram illustrates the theory of the NIR face recognition system and our proposed attack strategy. The left subfigure shows that infrared-absorbing ink can absorb diffracted NIR light and reduce the reflected NIR light used for imaging. The right subfigure presents our adversarial ink patch. The VIS image, perturbed with the ink, remains imperceptible, while the NIR image successfully executes an adversarial attack.
• Figure 2: An overview of generating NIR face recognition adversarial patches using the Differential Evolution (DE) framework: The process starts with an initial population of shapes, which undergo shape and position exploration to generate a child population. A fitness function evaluates both parent and child populations, selecting the best individuals to enhance adversarial effectiveness. Finally, the optimal individual is then applied to the NIR image for the attack.
• Figure 3: The shape and position optimization with constraint process is illustrated as follows: Subfigure (a) depicts the evolution of the shape and its position. Subfigure (b) shows the updated shape after the adjustment in (a), which exceeds the parameter boundaries. Subfigure (c) presents the corrected shape derived from (b). Finally, Subfigure (d) highlights the valid area, denoted as $\mathcal{P}$, constrained within $M^F$.
• Figure 4: Experiment preparation for our method and AiD cohen2023accessorize
• Figure 5: Examples of digital attacks using ink-shaping attack. For each group, the five images correspond to the gallery VIS image, the unattacked original image, the VIS and NIR image after the attacks, and the VIS image corresponding to the predicted wrong class after the attacks. Black text denotes the original correct ID and its probability, while red text indicates the misclassified ID and its confidence after the attack.