Detecting Adversarial Data using Perturbation Forgery

TL;DR

This work tackles the poor generalization of adversarial detectors to unseen attacks by modeling adversarial noise as proximal Gaussian distributions and proving the existence of an open covering of such distributions under the Wasserstein metric $W$. It introduces Perturbation Forgery, a detector-training framework that creates this open covering through noise distribution perturbation, sparse masking, and pseudo-adversarial data production. By training on the covering, the detector achieves robust, model-agnostic detection against gradient-based, generative-based, and even physical attacks, with only modest inference overhead. Extensive experiments on general and face datasets demonstrate strong generalization across attacks, supported by ablations and analyses that emphasize the role of high-frequency sparse masking and distribution estimation.

Abstract

As a defense strategy against adversarial attacks, adversarial detection aims to identify and filter out adversarial data from the data flow based on discrepancies in distribution and noise patterns between natural and adversarial data. Although previous detection methods achieve high performance in detecting gradient-based adversarial attacks, new attacks based on generative models with imbalanced and anisotropic noise patterns evade detection. Even worse, the significant inference time overhead and limited performance against unseen attacks make existing techniques impractical for real-world use. In this paper, we explore the proximity relationship among adversarial noise distributions and demonstrate the existence of an open covering for these distributions. By training on the open covering of adversarial noise distributions, a detector with strong generalization performance against various types of unseen attacks can be developed. Based on this insight, we heuristically propose Perturbation Forgery, which includes noise distribution perturbation, sparse mask generation, and pseudo-adversarial data production, to train an adversarial detector capable of detecting any unseen gradient-based, generative-based, and physical adversarial attacks. Comprehensive experiments conducted on multiple general and facial datasets, with a wide spectrum of attacks, validate the strong generalization of our method.

Figures (4)

• Figure 1: Illustration of the proximity of adversarial noise distributions and perturbed noise distributions. Left: all the distributions of adversarial noise and the perturbed distributions are in a $\varepsilon$-ball centered on a given adversarial noise distribution. Right: by continuously perturbing the given adversarial noise distribution, we obtain an open covering of distributions of adversarial noise.
• Figure 2: Illustration of Perturbation Forgery. Before training, we estimate the noise distribution from a commonly used attack, then continuously perturb it in each batch to create an open covering of the adversarial noise distributions. Next, noises sampled from these perturbed distributions are converted into localized noise by applying sparse masks. Finally, pseudo-adversarial data is generated by adding these localized noises to natural samples.
• Figure 3: 2D T-SNE visualizations. (a) ImageNet100 flattened noises of adversarial data and Perturbation Forgery-generated data. (b) ImageNet100 features extracted by the backbone model trained with Perturbation Forgery.
• Figure 4: Impact of the initial attacks (AUROC) on CIFAR-10 under $\epsilon=4/255$. Y-axis: initial attack. X-axis: testing attack.

Theorems & Definitions (2)

• Definition 1: Proximal Noise Distribution
• Theorem 1