Improving the Transferability of Adversarial Attacks on Face Recognition with Diverse Parameters Augmentation

TL;DR

The paper tackles the problem of limited transferability in adversarial attacks on face recognition (FR) systems. It introduces Diverse Parameters Augmentation (DPA), a two-stage framework comprising Diverse Parameters Optimization (DPO) to create a large, diverse surrogate-model ensemble by mixing pre-trained and random initializations and preserving intermediate checkpoints, and Hard Model Aggregation (HMA) to apply beneficial perturbations to feature maps and ensemble hard models for higher transferability. The authors formalize the black-box FR attack objective with a distance metric $\mathcal{D}$ and constraint $\|\mathbf{x}^{adv}-\mathbf{x}^s\|_p \le \epsilon$, and demonstrate that DPA substantially improves attack success rates across LFW and CelebA-HQ on multiple FR backbones, including adversarially trained models, under JPEG compression and other settings. This work offers a novel parameter-augmentation perspective for evaluating FR vulnerabilities, with practical implications for both adversarial robustness research and the development of more resilient FR systems.

Abstract

Face Recognition (FR) models are vulnerable to adversarial examples that subtly manipulate benign face images, underscoring the urgent need to improve the transferability of adversarial attacks in order to expose the blind spots of these systems. Existing adversarial attack methods often overlook the potential benefits of augmenting the surrogate model with diverse initializations, which limits the transferability of the generated adversarial examples. To address this gap, we propose a novel method called Diverse Parameters Augmentation (DPA) attack method, which enhances surrogate models by incorporating diverse parameter initializations, resulting in a broader and more diverse set of surrogate models. Specifically, DPA consists of two key stages: Diverse Parameters Optimization (DPO) and Hard Model Aggregation (HMA). In the DPO stage, we initialize the parameters of the surrogate model using both pre-trained and random parameters. Subsequently, we save the models in the intermediate training process to obtain a diverse set of surrogate models. During the HMA stage, we enhance the feature maps of the diversified surrogate models by incorporating beneficial perturbations, thereby further improving the transferability. Experimental results demonstrate that our proposed attack method can effectively enhance the transferability of the crafted adversarial face examples.

Figures (8)

• Figure 1: Top: comparison between traditional augmentation-based adversarial attack methods and our proposed method. The black pattern filling on the left and right sides of the blue line represents input-based and parameter-based augmentation, respectively. The orange pattern filling indicates feature-based augmentation. Bottom: comparison of performance among 4 types of augmentations.
• Figure 2: The framework of the Diverse Parameters Optimization (DPO). We enhance the diversity of the surrogate model parameters by integrating both pre-trained and random initializations. The method yields a diverse set of surrogate model parameters, which enhances the parameter diversity of the surrogate FR models and consequently improves transferability of the crafted adversarial examples.
• Figure 3: The framework of the Hard Model Aggregation (HMA). After acquiring a surrogate model set with diverse parameters (i.e., ${\mathbf{V}}^q_c$), we introduce beneficial perturbations with the optimization direction opposite to that of adversarial perturbations onto the feature maps of these diversified surrogate models, transforming them into hard models and aggregate the hard models to increase the transferability.
• Figure 4: The illustration of adversarial examples crafted by various attacks. First column: some source images. Last column: the corresponding target images. The second to fifth columns exhibit the corresponding adversarial face examples crafted by SIA sia, BPFA bpfa, BSR bsr, and Our proposed attack, respectively.
• Figure 5: Performance of ASR across various JPEG Q values: (a) Results on the LFW dataset. (b) Results on the CelebA-HQ dataset.