Evaluating Adversarial Robustness in the Spatial Frequency Domain

TL;DR

This work addresses the vulnerability of CNNs to adversarial perturbations by introducing a Spatial Frequency (SF) layer that computes a block-wise DCT spectrum on $8\times8$ image blocks, producing a rich $192$-channel frequency representation that replaces early feature extractors in canonical backbones. By evaluating SF-CNN variants (e.g., SF-ResNet18, SF-DenseNet, SF-EfficientNet, SF-VGG11) on multiple datasets, the authors show superior robustness to white-box and transfer attacks, and they reveal that robustness largely stems from leveraging low-frequency components, as demonstrated by interpolation and substitution analyses with $C_{88}$-style layers. The study also contrasts pixel-domain and frequency-domain perturbations, noting that frequency-domain attacks can be detected via shifts in frequency distributions and that SF-CNNs maintain more stable attention patterns under attack, as shown by Grad-CAM visualizations. These findings suggest that incorporating spatial-frequency features into CNNs can guide robust model design and enable effective defenses against adversarial perturbations, with practical implications for safety-critical vision systems.

Abstract

Convolutional Neural Networks (CNNs) have dominated the majority of computer vision tasks. However, CNNs' vulnerability to adversarial attacks has raised concerns about deploying these models to safety-critical applications. In contrast, the Human Visual System (HVS), which utilizes spatial frequency channels to process visual signals, is immune to adversarial attacks. As such, this paper presents an empirical study exploring the vulnerability of CNN models in the frequency domain. Specifically, we utilize the discrete cosine transform (DCT) to construct the Spatial-Frequency (SF) layer to produce a block-wise frequency spectrum of an input image and formulate Spatial Frequency CNNs (SF-CNNs) by replacing the initial feature extraction layers of widely-used CNN backbones with the SF layer. Through extensive experiments, we observe that SF-CNN models are more robust than their CNN counterparts under both white-box and black-box attacks. To further explain the robustness of SF-CNNs, we compare the SF layer with a trainable convolutional layer with identical kernel sizes using two mixing strategies to show that the lower frequency components contribute the most to the adversarial robustness of SF-CNNs. We believe our observations can guide the future design of robust CNN models.

Figures (10)

• Figure 1: The comparison between SF-CNNs and standard CNNs. Standard CNNs utilize $N$ trainable convolutional layers in the feature extractor and backbone classifier. SF-CNNs replace the feature extractor with the SF layer which conducts the block-wise DCT to extract the spatial frequencies as the initial image features but follows the same $M$-layer backbone architecture for classification.
• Figure 2: Examples of adversarial attacks in the pixel or the frequency domain with $\epsilon$ and $\epsilon_f$ set to $0.01$. Perturbation on the frequency domain leaves a more visible adversarial noise with a periodic pattern.
• Figure 3: Grad-Cam visualizations of transfer attacks from VGG11. Top, middle, and bottom rows present the images, Grad-Cam visualization of ResNet18, and that of SF-ResNet18, respectively.
• Figure 4: Grad-Cam visualizations of transfer attacks from SF-VGG11. Top, middle, and bottom rows present the images, Grad-Cam visualization of ResNet18, and that of SF-ResNet18, respectively.
• Figure 5: Examples of the original image (top), lfr (middle) and hfr (bottom). Despite using only the lowest frequency components, lfrs are visually more similar to the original images than hfrs.