Robust Feature-Level Adversaries are Interpretability Tools

TL;DR

This paper introduces feature-level adversaries that perturb latent representations of generative models to create perceptible, interpretable attacks on vision systems. It defines a differentiable pipeline with patch, region, and generalized-patch attacks, augmented by disguise regularizers to produce attacks that are both effective and interpretable, even at ImageNet scale. The authors demonstrate robust, universal, disguised, physically-realizable, and black-box attack capabilities and show how these adversaries illuminate network representations via copy/paste interpretability tests. They also position these attacks as practical tools for diagnosing model weaknesses and guiding future defenses and adversarial training.

Abstract

The literature on adversarial attacks in computer vision typically focuses on pixel-level perturbations. These tend to be very difficult to interpret. Recent work that manipulates the latent representations of image generators to create "feature-level" adversarial perturbations gives us an opportunity to explore perceptible, interpretable adversarial attacks. We make three contributions. First, we observe that feature-level attacks provide useful classes of inputs for studying representations in models. Second, we show that these adversaries are uniquely versatile and highly robust. We demonstrate that they can be used to produce targeted, universal, disguised, physically-realizable, and black-box attacks at the ImageNet scale. Third, we show how these adversarial images can be used as a practical interpretability tool for identifying bugs in networks. We use these adversaries to make predictions about spurious associations between features and classes which we then test by designing "copy/paste" attacks in which one natural image is pasted into another to cause a targeted misclassification. Our results suggest that feature-level attacks are a promising approach for rigorous interpretability research. They support the design of tools to better understand what a model has learned and diagnose brittle feature associations. Code is available at https://github.com/thestephencasper/feature_level_adv

Figures (15)

• Figure 1: Our feature-level adversaries are useful for interpreting deep networks (we used a ResNet50 he2016deep). (a) A pixel-level adversarial patch trained to make images of bees misclassified as flies. (b) An analogous feature-level adversarial patch. (c) A correctly classified image of a bee. (d) A successful copy/paste attack whose design was guided by adversarial examples like the one in (b).
• Figure 2: Our fully-differentiable pipeline for creating feature-level attacks. In each experiment, we create either "patch," "region," or "generalized patch" attacks. The regularization terms in the loss based on an external classifier and discriminator are optional and are meant to make the inserted feature appear disguised as some non-target class.
• Figure 3: Examples of targeted, universal feature-level adversaries from patch (top), region (middle), and generalized patch (bottom) attacks. The first four columns show the adversarial features. The mean target class confidence is labeled 'Adv.' and is calculated under random source images (and random insertion locations for patch and generalized patch attacks). The target network's disguise class confidence for each patch or extracted generalized patch is labeled 'Disg.' The final column shows examples of the features applied to images. The example image for each is labeled with its source and target class confidences.
• Figure 4: Targeted, universal patch attacks compared. Successful disguise success rate (x axis) shows the proportion of attacks in which the patch was not classified by the network as the target class when viewed on its own. Mean target class confidence (y axis) gives the empirical target class confidences of 250 patch attacks. Each is an average over 100 source images. The proportion of each distribution above 0.5 gives a lower bound for the top-1 attack success rate. The mean target class confidence for using randomly sampled natural target class images as patches is 0.0024 and is shown as a thin dotted line at the bottom.
• Figure 5: Examples of targeted, disguised, universal, and physically-realizable feature-level attacks. See Appendix \ref{['app:printable_examples']} Fig. \ref{['fig:all_printable']} for full-sized versions of the patches.