Interpreting Adversarial Examples by Activation Promotion and Suppression

TL;DR

This work probes the interpretability of adversarial examples by proposing a promotion-suppression framework (PSE) that categorizes pixel perturbations based on their influence on true and target class logits. It connects pixel-level perturbations to image-level discriminative regions via CAM and to hidden-unit semantics via network dissection, revealing a tight link between perturbation sensitivity and concept-level interpretability. The study demonstrates that perturbations often act on discriminative regions corresponding to true- and target-class concepts, and introduces refinement strategies guided by CAM to improve attack efficiency and robustness. Overall, the paper provides a multi-level interpretation of adversarial effects and suggests interpretability-informed defenses, such as neuron masking, to bolster robustness with manageable costs to clean accuracy.

Abstract

It is widely known that convolutional neural networks (CNNs) are vulnerable to adversarial examples: images with imperceptible perturbations crafted to fool classifiers. However, interpretability of these perturbations is less explored in the literature. This work aims to better understand the roles of adversarial perturbations and provide visual explanations from pixel, image and network perspectives. We show that adversaries have a promotion-suppression effect (PSE) on neurons' activations and can be primarily categorized into three types: i) suppression-dominated perturbations that mainly reduce the classification score of the true label, ii) promotion-dominated perturbations that focus on boosting the confidence of the target label, and iii) balanced perturbations that play a dual role in suppression and promotion. We also provide image-level interpretability of adversarial examples. This links PSE of pixel-level perturbations to class-specific discriminative image regions localized by class activation mapping (Zhou et al. 2016). Further, we examine the adversarial effect through network dissection (Bau et al. 2017), which offers concept-level interpretability of hidden units. We show that there exists a tight connection between the units' sensitivity to adversarial attacks and their interpretability on semantic concepts. Lastly, we provide some new insights from our interpretation to improve the adversarial robustness of networks.

Figures (18)

• Figure 1: Explanation of adversarial perturbations produced by the C&W attack carlini2017towards. The first column shows the original image (with true label 'Japanese spaniel') and its adversarial example (with target label 'bullfrog'). The second column demonstrates CAM of the original image with respect to the true label and CAM of the adversarial example with respect to the target label. At the third column, the adversarial perturbations are overlaid on CAM, and their effects are categorized by our approach: suppression-dominated perturbations (white, at the face of spaniel), promotion-dominated perturbations (black, at the face of bullfrog), and balance-dominated perturbations (gray).
• Figure 2: Illustration on sensitivity measure via the 'badger'-to-'computer' adversarial example generated by C&W attack. Here the true label is 'badger' and the target label is 'computer'. The first column shows the adversarial example and the heat map of $\ell_2$-norm distortion at each grid region, i.e., $\{ \| \boldsymbol{\delta}_{\mathcal{G}_i} \|_2 \}$. The second column presents $\ell_p$ norm of $\boldsymbol{\delta}_{\mathcal{G}_i}$ ($p = 2,\infty$), sensitivity scores $d_{0,i}$ and $d_{t,i}$, and PSR $r_i$ versus the index of grid regions, where the dash lines correspond to the PSR threshold $\pm 1$.
• Figure 3: Correlation between sensitivity scores $\{ s_i \}$ and $\ell_2$ distortion values $\{ \| \boldsymbol{\delta}_{\mathcal{G}_i} \|_2 \}$. Left: Pearson correlation. Right: Kendall rank correlation.
• Figure 4: Visualizing CAMs of natural image and its adversarial example (generated by C&W attack) w.r.t. the true label 'badger' and the target label 'computer', respectively. The heat map color from blue to red represents the least and the most discriminative region localized by CAM, respectively. Here the values of CAMs are normalized w.r.t. the largest value cross CAMs.
• Figure 5: IS under $4$ attack types & $3$ visual explanation methods on Resnet. Left: IS defined on $F(\mathbf x_0, t_0)$. Right: IS defined on $F(\mathbf x^\prime, t)$. Each box plot represents IS values of $5000$ natural/adversarial examples from ImageNet.

Theorems & Definitions (1)

• Definition 1