Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Understanding Adversarial Robustness from Feature Maps of Convolutional Layers

Cong Xu, Wei Zhang, Jun Wang, Min Yang

TL;DR

The paper tackles adversarial robustness by linking it to the anti-perturbation ability encoded in convolutional feature maps. It shows theoretically that larger feature maps before average pooling boost robustness, while max pooling can undermine it, and proposes two simple, practical modifications—upsampling inputs and shrinking downsampling strides—to enlarge feature maps. Empirically, these changes improve both natural accuracy and robustness across a range of CNNs and ViTs, and are complementary to existing defense methods. The work offers a new architectural perspective on robustness, suggesting that careful feature-map design can yield robust and more capable models with minimal modification effort.

Abstract

The adversarial robustness of a neural network mainly relies on two factors: model capacity and anti-perturbation ability. In this paper, we study the anti-perturbation ability of the network from the feature maps of convolutional layers. Our theoretical analysis discovers that larger convolutional feature maps before average pooling can contribute to better resistance to perturbations, but the conclusion is not true for max pooling. It brings new inspiration to the design of robust neural networks and urges us to apply these findings to improve existing architectures. The proposed modifications are very simple and only require upsampling the inputs or slightly modifying the stride configurations of downsampling operators. We verify our approaches on several benchmark neural network architectures, including AlexNet, VGG, RestNet18, and PreActResNet18. Non-trivial improvements in terms of both natural accuracy and adversarial robustness can be achieved under various attack and defense mechanisms. The code is available at \url{https://github.com/MTandHJ/rcm}.

Understanding Adversarial Robustness from Feature Maps of Convolutional Layers

TL;DR

The paper tackles adversarial robustness by linking it to the anti-perturbation ability encoded in convolutional feature maps. It shows theoretically that larger feature maps before average pooling boost robustness, while max pooling can undermine it, and proposes two simple, practical modifications—upsampling inputs and shrinking downsampling strides—to enlarge feature maps. Empirically, these changes improve both natural accuracy and robustness across a range of CNNs and ViTs, and are complementary to existing defense methods. The work offers a new architectural perspective on robustness, suggesting that careful feature-map design can yield robust and more capable models with minimal modification effort.

Abstract

The adversarial robustness of a neural network mainly relies on two factors: model capacity and anti-perturbation ability. In this paper, we study the anti-perturbation ability of the network from the feature maps of convolutional layers. Our theoretical analysis discovers that larger convolutional feature maps before average pooling can contribute to better resistance to perturbations, but the conclusion is not true for max pooling. It brings new inspiration to the design of robust neural networks and urges us to apply these findings to improve existing architectures. The proposed modifications are very simple and only require upsampling the inputs or slightly modifying the stride configurations of downsampling operators. We verify our approaches on several benchmark neural network architectures, including AlexNet, VGG, RestNet18, and PreActResNet18. Non-trivial improvements in terms of both natural accuracy and adversarial robustness can be achieved under various attack and defense mechanisms. The code is available at \url{https://github.com/MTandHJ/rcm}.
Paper Structure (22 sections, 1 theorem, 23 equations, 13 figures, 11 tables)

This paper contains 22 sections, 1 theorem, 23 equations, 13 figures, 11 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Robust learning strategies
  4. Robust network architectures
  5. Anti-Perturbation Ability of Convolutional Feature Maps
  6. Preliminaries
  7. Connection between anti-perturbation ability and convolutional feature maps
  8. Feasible Ways to Enlarge Feature maps
  9. Upsampling inputs
  10. Shrinking sliding strides
  11. Experiments
  12. Experimental setup
  13. Upsampling inputs versus shrinking sliding strides
  14. Average pooling versus max pooling
  15. Improvements on various neural network architectures
  16. ...and 7 more sections

Key Result

Proposition 1

For any sample $x$, let $g(x) \in \mathbb{R}^{H \times W}$ denote the corresponding convolutional feature map. Assume that the random perturbation $\delta$ follows a probability distribution $\mathcal{D}$ such that the elements of $\Delta(\delta) = g(x+\delta)-g(x)$ are independent from each other a Then Eq. staIneq holds true when $\mathcal{P}$ is average pooling and or when $\mathcal{P}$ is max

Figures (13)

  • Figure 1: Natural accuracy (%) and adversarial robustness (%) on CIFAR-10. Different defense mechanisms are applied to the baseline and improved ResNet18.
  • Figure 2: The $\ell_{\infty}$ norm of input perturbations from $\mathcal{U}[-0.1, 0.1]$ and the resulting disturbances at each layer. Left: A randomly initialized CNN followed by average pooling; Right: A randomly initialized CNN followed by max pooling.
  • Figure 3: (a) Upsampling the input using nearest interpolation with different scales. (b) An illustration of feature redundancy caused by unsampling.
  • Figure 4: The architecture of ResNet with the $s_1$-$s_2$-$s_3$-$s_4$ sliding stride configuration. It consists of 2 fixed terminals and 4 intermediate stages. Each stage contains several convolutional layers.
  • Figure 5: Natural accuracy (%) comparisons of ResNet18 and its modified versions on CIFAR-10. (a) Upsampling using bilinear and nearest interpolation, or learnable deconvolution under different scales; (b) Shrinking sliding strides using various configurations.
  • ...and 8 more figures

Theorems & Definitions (7)

  • Remark 1
  • Proposition 1
  • proof
  • Remark 2
  • Remark 3
  • Remark 4
  • Remark 5