Investigating Adversarial Robustness against Preprocessing used in Blackbox Face Recognition

TL;DR

This work investigates how preprocessing steps in face recognition (detection and downsampling) affect the transferability of adversarial examples in blackbox settings. It analyzes three state-of-the-art attacks (LowKey, MIM, TIP-IM) across eleven face detectors and four interpolation methods, revealing that detector choice can drastically reduce attack effectiveness (up to 78% ASR reduction), while interpolation has a smaller impact. The authors propose a preprocessing-invariant attack method based on an ensemble of preprocessing functions, which improves transferability by up to 27% without compromising perceptual similarity. The findings highlight the critical role of preprocessing in FR security and motivate incorporating preprocessing-aware defenses to improve adversarial generalization in real-world systems.

Abstract

Face Recognition (FR) models have been shown to be vulnerable to adversarial examples that subtly alter benign facial images, exposing blind spots in these systems, as well as protecting user privacy. End-to-end FR systems first obtain preprocessed faces from diverse facial imagery prior to computing the similarity of the deep feature embeddings. Whilst face preprocessing is a critical component of FR systems, and hence adversarial attacks against them, we observe that this preprocessing is often overlooked in blackbox settings. Our study seeks to investigate the transferability of several out-of-the-box state-of-the-art adversarial attacks against FR when applied against different preprocessing techniques used in a blackbox setting. We observe that the choice of face detection model can degrade the attack success rate by up to 78%, whereas choice of interpolation method during downsampling has relatively minimal impacts. Furthermore, we find that the requirement for facial preprocessing even degrades attack strength in a whitebox setting, due to the unintended interaction of produced noise vectors against face detection models. Based on these findings, we propose a preprocessing-invariant method using input transformations that improves the transferability of the studied attacks by up to 27%. Our findings highlight the importance of preprocessing in FR systems, and the need for its consideration towards improving the adversarial generalisation of facial adversarial examples.

Figures (4)

• Figure 1: a) The structure of a face recognition system kortli2020face. b) Example effects on adversarial FR attacks against different preprocessing methods used in blackbox FR systems, assuming a verification threshold of $> 0.15$.
• Figure 2: Different crops and their resulting Grad-CAM localisation maps for the ArcFace FR model.
• Figure 3: Heatmap of Intersection over Union (IoU) for different face detection models on the CelebA-HQ dataset.
• Figure 4: Visual comparison of information loss for different interpolation methods used when downsampling a LowKey adversarial example.