IRAD: Implicit Representation-driven Image Resampling against Adversarial Attacks

TL;DR

This work introduces IRAD, a novel image resampling defense against adversarial attacks that leverages an implicit continuous representation and a learned SampleNet to generate input-specific pixel shifts. By reconstructing a continuous scene and adaptively sampling pixels, IRAD breaks adversarial textures while preserving semantic content, addressing limitations of naive resampling. The approach demonstrates strong robustness across multiple models and datasets, and can accelerate diffusion-based purification methods with minimal loss of clean accuracy. IRAD thus provides a practical, flexible test-time defense with broad applicability and compatibility with existing purification techniques.

Abstract

We introduce a novel approach to counter adversarial attacks, namely, image resampling. Image resampling transforms a discrete image into a new one, simulating the process of scene recapturing or rerendering as specified by a geometrical transformation. The underlying rationale behind our idea is that image resampling can alleviate the influence of adversarial perturbations while preserving essential semantic information, thereby conferring an inherent advantage in defending against adversarial attacks. To validate this concept, we present a comprehensive study on leveraging image resampling to defend against adversarial attacks. We have developed basic resampling methods that employ interpolation strategies and coordinate shifting magnitudes. Our analysis reveals that these basic methods can partially mitigate adversarial attacks. However, they come with apparent limitations: the accuracy of clean images noticeably decreases, while the improvement in accuracy on adversarial examples is not substantial. We propose implicit representation-driven image resampling (IRAD) to overcome these limitations. First, we construct an implicit continuous representation that enables us to represent any input image within a continuous coordinate space. Second, we introduce SampleNet, which automatically generates pixel-wise shifts for resampling in response to different inputs. Furthermore, we can extend our approach to the state-of-the-art diffusion-based method, accelerating it with fewer time steps while preserving its defense capability. Extensive experiments demonstrate that our method significantly enhances the adversarial robustness of diverse deep models against various attacks while maintaining high accuracy on clean images.

Figures (6)

• Figure 1: Comparison of different sampling strategies based on the bilinear interpolation as the reconstruction method.
• Figure 2: Pipeline of the proposed IRAD.
• Figure 3: The PCA results.
• Figure 4: Case visualization of CIFAR10
• Figure 5: Case visualization of CIFAR100