Harmonizing Feature Maps: A Graph Convolutional Approach for Enhancing Adversarial Robustness

TL;DR

This work tackles the vulnerability of deep networks to adversarial perturbations by addressing feature contamination during inference, a gap left by conventional adversarial training. It introduces Feature Map-based Reconstructed Graph Convolution (FMR-GC), a plug-and-play module that builds a channel-wise graph among feature maps, uses graph convolution with a Laplacian-normalized adjacency, and calibrates perturbed features by aggregating neighborhood information. Empirical results on CIFAR-10/100 and Tiny ImageNet show consistent robustness improvements across PGD-AT, TRADES, and AWP, while preserving or even improving clean accuracy, and analyses reveal the importance of graph sparsity, node-feature representations, and early-insertion of FMR-GC. The method is scalable, integrates with existing defenses, and provides a principled framework to leverage inter-feature relationships for defense against adversarial attacks in practical settings.

Abstract

The vulnerability of Deep Neural Networks to adversarial perturbations presents significant security concerns, as the imperceptible perturbations can contaminate the feature space and lead to incorrect predictions. Recent studies have attempted to calibrate contaminated features by either suppressing or over-activating particular channels. Despite these efforts, we claim that adversarial attacks exhibit varying disruption levels across individual channels. Furthermore, we argue that harmonizing feature maps via graph and employing graph convolution can calibrate contaminated features. To this end, we introduce an innovative plug-and-play module called Feature Map-based Reconstructed Graph Convolution (FMR-GC). FMR-GC harmonizes feature maps in the channel dimension to reconstruct the graph, then employs graph convolution to capture neighborhood information, effectively calibrating contaminated features. Extensive experiments have demonstrated the superior performance and scalability of FMR-GC. Moreover, our model can be combined with advanced adversarial training methods to considerably enhance robustness without compromising the model's clean accuracy.

Figures (5)

• Figure 1: Visualization of feature activation maps via various training methods utilizing Grad-CAM selvaraju2017grad. The 'standard model' denotes a model trained on clean examples. Left-to-right: original image, the feature activation map of clean examples on the standard model, and the feature activation maps of adversarial examples on the standard model, PGD-AT, FPCM, FSR, and FMR-GC-AT.
• Figure 2: The process of reconstructing the graph at the channel level by exploiting the similarity between feature maps.
• Figure 3: The pipeline of our proposed model depicting the process. FMR-GC considers each feature map as a node and constructs a graph within the channel dimension using global average pooling and similarity computation. After reconstructing the graph, we perform graph convolution to calibrate contaminated features by leveraging contextual information.
• Figure 4: Analysis of the influence of reconstruction graph sparsity on model performance on CIFAR-10 using ResNet18. The $x$-axis represents the graph sparsity, while the $y$-axis represents the accuracy (%).
• Figure 5: Comparisons with varying $\epsilon$ values on CIFAR-10 using WRN34-10. The $x$-axis represents the $\epsilon$ value, while $y$-axis represents the robust accuracy (%).