Improving the JPEG-resistance of Adversarial Attacks on Face Recognition by Interpolation Smoothing

TL;DR

This work tackles the degradation of adversarial face attacks by JPEG compression and introduces the Interpolation Attack Method (IAM), which uses bilinear interpolation to downsample and then upsample perturbations during iterative attack generation to suppress high-frequency components. IAM can be integrated with existing gradient-based attacks and does not rely on knowledge of the JPEG quality factor ($QF$) while delivering improved JPEG-resistance and transferability on CelebA-HQ and LFW across multiple face-recognition models and JPEG levels. Experimental results demonstrate consistent improvements in attack success under JPEG compression, indicating a practical vulnerability of FR systems to interpolation-smoothed adversaries in typical image pipelines.

Abstract

JPEG compression can significantly impair the performance of adversarial face examples, which previous adversarial attacks on face recognition (FR) have not adequately addressed. Considering this challenge, we propose a novel adversarial attack on FR that aims to improve the resistance of adversarial examples against JPEG compression. Specifically, during the iterative process of generating adversarial face examples, we interpolate the adversarial face examples into a smaller size. Then we utilize these interpolated adversarial face examples to create the adversarial examples in the next iteration. Subsequently, we restore the adversarial face examples to their original size by interpolating. Throughout the entire process, our proposed method can smooth the adversarial perturbations, effectively mitigating the presence of high-frequency signals in the crafted adversarial face examples that are typically eliminated by JPEG compression. Our experimental results demonstrate the effectiveness of our proposed method in improving the JPEG-resistance of adversarial face examples.

Figures (4)

• Figure 1: Comparison of pixel values between the adversarial examples generated by BIM and our method. The horizontal axis represents the neighboring pixel points of adversarial examples, and the vertical axis represents the corresponding pixel values.
• Figure 2: Images and their corresponding DCT coefficients. The images in the first row represent the original image, adversarial examples generated by BIM and our method. The second row are their corresponding DCT coefficients. In the DCT domain, the upper left corner is the location of the lowest frequency, and the frequency gradually increases in other locations. Light color indicates larger coefficient and dark color indicates smaller coefficient.
• Figure 3: ASRs of the JPEG adversarial examples with different QF. The model before $\rightarrow$ is the attack model, and the model after $\rightarrow$ is the victim model.
• Figure 4: ASRs of the JPEG adversarial examples generated by BIM+IAM with different interpolation factors. In each subfigure, the subtitle is the corresponding attack model, the models in the legend are the corresponding victim models and the horizontal lines respresent the ASRs of BIM in the corresponding attack scenario.