Adversarial Samples Are Not Created Equal

TL;DR

This paper reframes adversarial vulnerability by distinguishing two distinct weaknesses: exploitation of non-robust data features and a separate vulnerability tied to sharpness in the loss landscape, termed adversarial bugs. It introduces an ensemble-based metric, $JS_\Delta$, to quantify whether an adversarial perturbation significantly manipulates data features, enabling a separation between feature-based attacks and adversarial bugs. Through CIFAR10 and SVHN experiments with various training regimes, it shows that larger perturbations tend to shift attacks toward non-robust features, while SAM reduces the occurrence of adversarial bugs but does not fundamentally increase robustness of data features. The work also re-evaluates robust datasets, revealing that non-robust features persist and that robustness gaps arise from naive distillation of robust features, offering a nuanced view of how to evaluate and improve adversarial robustness in practice.

Abstract

Over the past decade, numerous theories have been proposed to explain the widespread vulnerability of deep neural networks to adversarial evasion attacks. Among these, the theory of non-robust features proposed by Ilyas et al. has been widely accepted, showing that brittle but predictive features of the data distribution can be directly exploited by attackers. However, this theory overlooks adversarial samples that do not directly utilize these features. In this work, we advocate that these two kinds of samples - those which use use brittle but predictive features and those that do not - comprise two types of adversarial weaknesses and should be differentiated when evaluating adversarial robustness. For this purpose, we propose an ensemble-based metric to measure the manipulation of non-robust features by adversarial perturbations and use this metric to analyze the makeup of adversarial samples generated by attackers. This new perspective also allows us to re-examine multiple phenomena, including the impact of sharpness-aware minimization on adversarial robustness and the robustness gap observed between adversarially training and standard training on robust datasets.

Figures (12)

• Figure 1: Illustration of the decision boundary for a deep neural network, trained on the task of binary classification, that allows for (a) adversarial samples that utilize non-robust features and (b) adversarial bugs. The former exist because the classifier used predictive features of the data distribution that are not human-aligned, while the latter appear as irregular blind spots of a particular model instance.
• Figure 2: Normalized histograms of $JS_\Delta$ for adversarial samples of a non-robust ResNet50 model trained on CIFAR10. All samples are generated via a targeted PGD-100 attack. We observe that for low magnitudes of perturbation, a large percentage of adversarial samples can be identified as adversarial bugs. For larger magnitudes of perturbation, the attacker is able to create manipulate non-robust features in the majority of adversarial samples.
• Figure 3: Normalized histograms of $JS_\Delta$ for successful adversarial samples of a robust ResNet50 model adversarially trained on CIFAR10. All samples are generated via a targeted PGD-100 attack. We observe that larger attack perturbations are needed to create both adversarial bugs and adversarial samples that utilize non-robust features.
• Figure 4: Loss landscape surrounding a natural test sample (left) and adversarial test samples with large (middle) and small (right) $JS_\Delta$ for a ResNet50 model trained on CIFAR10. We find that the loss of adversarial bugs (right) displays a sharp minimum, while the loss for adversarial samples that manipulate data features (middle) have curvature comparable to that of natural samples.
• Figure 5: Normalized histograms of $JS_\Delta$ for adversarial samples of a ResNet50 model trained via SAM ($\rho=0.3$) on CIFAR10. All samples are generated via a targeted PGD-100 attack. We observe a notable decline in adversarial bugs as compared to its SGD trained counterpart (Figure \ref{['fig:cifar_hist']}).