Feature Denoising for Improving Adversarial Robustness

TL;DR

The paper addresses adversarial vulnerability in image classification by showing that adversarial perturbations induce noise in intermediate feature maps and proposing feature-denoising blocks that operate on those features. These blocks, trained end-to-end with adversarial examples, incorporate non-local means and other filters to suppress perturbations, yielding state-of-the-art robustness on ImageNet against strong white-box and black-box attacks, including a CAAD 2018 defense win. Ablation studies reveal non-local denoising as particularly effective, with critical design elements like a 1×1 convolution and a residual connection essential for training stability and performance. While there is a tradeoff with clean-image accuracy, the approach demonstrates a path toward innate adversarial robustness via architectural design rather than post hoc preprocessing.

Abstract

Adversarial attacks to image classification systems present challenges to convolutional networks and opportunities for understanding them. This study suggests that adversarial perturbations on images lead to noise in the features constructed by these networks. Motivated by this observation, we develop new network architectures that increase adversarial robustness by performing feature denoising. Specifically, our networks contain blocks that denoise the features using non-local means or other filters; the entire networks are trained end-to-end. When combined with adversarial training, our feature denoising networks substantially improve the state-of-the-art in adversarial robustness in both white-box and black-box attack settings. On ImageNet, under 10-iteration PGD white-box attacks where prior art has 27.9% accuracy, our method achieves 55.7%; even under extreme 2000-iteration PGD white-box attacks, our method secures 42.6% accuracy. Our method was ranked first in Competition on Adversarial Attacks and Defenses (CAAD) 2018 --- it achieved 50.6% classification accuracy on a secret, ImageNet-like test dataset against 48 unknown attackers, surpassing the runner-up approach by ~10%. Code is available at https://github.com/facebookresearch/ImageNet-Adversarial-Training.

Figures (8)

• Figure 1: Feature map in the res$_3$ block of an ImageNet-trained ResNet-50 He2016 applied on a clean image (top) and on its adversarially perturbed counterpart (bottom). The adversarial perturbation was produced using PGD Madry2018 with maximum perturbation $\epsilon\!=\!$ 16 (out of 256). In this example, the adversarial image is incorrectly recognized as "space heater"; the true label is "digital clock".
• Figure 2: More examples similar to Figure \ref{['fig:teaser']}. We show feature maps corresponding to clean images (top) and to their adversarial perturbed versions (bottom). The feature maps for each pair of examples are from the same channel of a res$_3$ block in the same ResNet-50 trained on clean images. The attacker has a maximum perturbation $\epsilon=16$ in the pixel domain.
• Figure 3: Adversarial images and their feature maps before (left) and after (right) the denoising operation (blue box in Figure \ref{['fig:block']}). Here each pair of feature maps are from the same channel of a res$_3$ block in the same adversarially trained ResNet-50 equipped with (Gaussian) non-local means denoising blocks. The attacker has a maximum perturbation $\epsilon\!=\!16$ for each pixel.
• Figure 4: A generic denoising block. It wraps the denoising operation (e.g., non-local means, bilateral, mean, median filters) with a 1$\times$1 convolution and an identity skip connection He2016.
• Figure 5: A block with non-local means as its denoising operation. The blue part illustrates the implementation of non-local means in Eqn. (\ref{['eq:nonlocal']}). The shapes of the feature tensors are noted, with corresponding reshaping/transposing performed: here, $H$ and $W$ are the height and width of the feature maps, and we use 256 channels as an example. If softmax is used, it is the Gaussian version (with appropriate 1$\times$1 convolution embeddings used; omitted in this figure); if softmax is not used, it is the dot product version.