Diffusion-based Adversarial Purification from the Perspective of the Frequency Domain

TL;DR

This paper reframes diffusion-based adversarial purification in the frequency domain, showing that adversarial perturbations increasingly disrupt both amplitude and phase spectra with diffusion time. It introduces FreqPure, which at each reverse step preserves low-frequency amplitude and constrains low-frequency phase to serve as a robust prior for reconstructing high-frequency details. The approach yields superior robustness against strong adversarial attacks across CIFAR-10, SVHN, and ImageNet, with quantitative and qualitative evidence that purified images resemble clean originals more closely than baselines. The work underscores the value of frequency-aware priors in purification and points to gradient-computation challenges as an avenue for future refinement.

Abstract

The diffusion-based adversarial purification methods attempt to drown adversarial perturbations into a part of isotropic noise through the forward process, and then recover the clean images through the reverse process. Due to the lack of distribution information about adversarial perturbations in the pixel domain, it is often unavoidable to damage normal semantics. We turn to the frequency domain perspective, decomposing the image into amplitude spectrum and phase spectrum. We find that for both spectra, the damage caused by adversarial perturbations tends to increase monotonically with frequency. This means that we can extract the content and structural information of the original clean sample from the frequency components that are less damaged. Meanwhile, theoretical analysis indicates that existing purification methods indiscriminately damage all frequency components, leading to excessive damage to the image. Therefore, we propose a purification method that can eliminate adversarial perturbations while maximizing the preservation of the content and structure of the original image. Specifically, at each time step during the reverse process, for the amplitude spectrum, we replace the low-frequency components of the estimated image's amplitude spectrum with the corresponding parts of the adversarial image. For the phase spectrum, we project the phase of the estimated image into a designated range of the adversarial image's phase spectrum, focusing on the low frequencies. Empirical evidence from extensive experiments demonstrates that our method significantly outperforms most current defense methods.

Figures (13)

• Figure 1: We decompose the image into the amplitude spectrum (left) and the phase spectrum (right), and calculate the differences between the adversarial images and the original images, respectively. The damage caused by adversarial perturbations tends to increase monotonically with frequency for both spectra.
• Figure 2: Pipeline of our method. The core of our method is to preserve, at each time-step of the reverse process, the amplitude and phase spectrum information of the original clean samples extracted from adversarial images as a prior. This ensures the retention of the original image's content and structural information while also providing guidance for the restoration of high-frequency details.
• Figure 3: Visualization of origianl clean images , adversarial images and purified images. The images purified by our method are most similar to the origianl clean images.
• Figure 4: Joint distribution of the original images and purified images. The distributions of the purified images by our method and the original images are the most similar.
• Figure 5: Robust and Standard Accuracy under different thresholds $D_A$ (left) and under different combinations of $D_P$ and $\delta$ (right).

Theorems & Definitions (8)

• Definition 3.1
• Theorem 3.2
• Definition 3.3
• Theorem 3.4
• Theorem 4.1
• proof
• Theorem 4.2
• proof