Learning to Transform Dynamically for Better Adversarial Transferability

TL;DR

The paper tackles adversarial transferability by addressing the limitations of fixed input transformations. It introduces Learning to Transform (L2T), which casts the search for optimal transformation sequences as a trajectory optimization problem and solves it with reinforcement learning to adapt transformations at each attack iteration. By sampling sequences from a learnable distribution over a pool of operations and updating this distribution via gradient ascent, L2T achieves superior transferability across diverse surrogate/target models, defenses, and vision APIs on ImageNet, including high-impact real-world systems like GPT-4V and Bard. The results indicate that dynamic, per-iteration transformation selection substantially enhances attack effectiveness and informs defense strategies against transferable adversaries.

Abstract

Adversarial examples, crafted by adding perturbations imperceptible to humans, can deceive neural networks. Recent studies identify the adversarial transferability across various models, \textit{i.e.}, the cross-model attack ability of adversarial samples. To enhance such adversarial transferability, existing input transformation-based methods diversify input data with transformation augmentation. However, their effectiveness is limited by the finite number of available transformations. In our study, we introduce a novel approach named Learning to Transform (L2T). L2T increases the diversity of transformed images by selecting the optimal combination of operations from a pool of candidates, consequently improving adversarial transferability. We conceptualize the selection of optimal transformation combinations as a trajectory optimization problem and employ a reinforcement learning strategy to effectively solve the problem. Comprehensive experiments on the ImageNet dataset, as well as practical tests with Google Vision and GPT-4V, reveal that L2T surpasses current methodologies in enhancing adversarial transferability, thereby confirming its effectiveness and practical significance. The code is available at https://github.com/RongyiZhu/L2T.

Figures (14)

• Figure 1: For input transformation-based attacks, most works design a fixed transformation and use it to craft the adversarial perturbation. The learning-based methods preliminarily predict augmentation strategies for current images for better adversarial transferability. These methods cannot respond to the distribution shifts between benign images and adversarial examples. We propose Learning to Transform (L2T), which uses the dynamic of the optimal transformation in each iteration to further boost the adversarial transferability.
• Figure 2: Comparsion for different operations in boosting the adversarial transferability. The number in the box denotes the number of fooled models (Maximum: 9). In (a), the horizontal axis denotes different transformation operations and the vertical axis denotes different benign examples. In (b), the vertical axis denotes the transformation used in the first iteration and the horizontal axis denotes the transformation used in the second iteration
• Figure 3: There exists an optimal transformation trajectory for boosting adversarial transferability. However, the search space increases exponentially with iteration number and operation number.
• Figure 4: Overview of the pipeline in L2T. We use probability in sampling $L$ transformations and update this probability through gradient ascent.
• Figure 5: Average attack success rates (%) of ten models on the adversarial examples crafted on each model. The x-axis of each sub-figure denotes different attack methods. We include the detail number in our supplementary material.