Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The Ultimate Combo: Boosting Adversarial Example Transferability by Composing Data Augmentations

Zebin Yun, Achi-Or Weingarten, Eyal Ronen, Mahmood Sharif

TL;DR

This work systematically studied how data augmentation affects transferability and identified augmentation combinations that help promote transferability, and theoretical analysis intuitively explains why certain augmentations promote transferability.

Abstract

To help adversarial examples generalize from surrogate machine-learning (ML) models to targets, certain transferability-based black-box evasion attacks incorporate data augmentations (e.g., random resizing). Yet, prior work has explored limited augmentations and their composition. To fill the gap, we systematically studied how data augmentation affects transferability. Specifically, we explored 46 augmentation techniques originally proposed to help ML models generalize to unseen benign samples, and assessed how they impact transferability, when applied individually or composed. Performing exhaustive search on a small subset of augmentation techniques and genetic search on all techniques, we identified augmentation combinations that help promote transferability. Extensive experiments with the ImageNet and CIFAR-10 datasets and 18 models showed that simple color-space augmentations (e.g., color to greyscale) attain high transferability when combined with standard augmentations. Furthermore, we discovered that composing augmentations impacts transferability mostly monotonically (i.e., more augmentations $\rightarrow$ $\ge$transferability). We also found that the best composition significantly outperformed the state of the art (e.g., 91.8% vs. $\le$82.5% average transferability to adversarially trained targets on ImageNet). Lastly, our theoretical analysis, backed by empirical evidence, intuitively explains why certain augmentations promote transferability.

The Ultimate Combo: Boosting Adversarial Example Transferability by Composing Data Augmentations

TL;DR

This work systematically studied how data augmentation affects transferability and identified augmentation combinations that help promote transferability, and theoretical analysis intuitively explains why certain augmentations promote transferability.

Abstract

To help adversarial examples generalize from surrogate machine-learning (ML) models to targets, certain transferability-based black-box evasion attacks incorporate data augmentations (e.g., random resizing). Yet, prior work has explored limited augmentations and their composition. To fill the gap, we systematically studied how data augmentation affects transferability. Specifically, we explored 46 augmentation techniques originally proposed to help ML models generalize to unseen benign samples, and assessed how they impact transferability, when applied individually or composed. Performing exhaustive search on a small subset of augmentation techniques and genetic search on all techniques, we identified augmentation combinations that help promote transferability. Extensive experiments with the ImageNet and CIFAR-10 datasets and 18 models showed that simple color-space augmentations (e.g., color to greyscale) attain high transferability when combined with standard augmentations. Furthermore, we discovered that composing augmentations impacts transferability mostly monotonically (i.e., more augmentations transferability). We also found that the best composition significantly outperformed the state of the art (e.g., 91.8% vs. 82.5% average transferability to adversarially trained targets on ImageNet). Lastly, our theoretical analysis, backed by empirical evidence, intuitively explains why certain augmentations promote transferability.
Paper Structure (22 sections, 3 theorems, 5 equations, 1 figure, 8 tables, 1 algorithm)

This paper contains 22 sections, 3 theorems, 5 equations, 1 figure, 8 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. Data Augmentations for Transferability
  4. Previous Attacks Leveraging Augmentations
  5. New Augmentations for Boosting Transferability
  6. Composing Augmentations
  7. Discovering Effective Compositions
  8. Theoretical Analysis of Augmentations
  9. Experimental Setup
  10. Experimental Results
  11. Individual Augmentations
  12. Exhaustive Search
  13. Genetic Search
  14. The Most Effective Combinations
  15. Parallel vs. Serial Composition
  16. ...and 7 more sections

Key Result

theorem 1

$\hat{f}$, attained by augmenting $\nabla_x J\left(x, y, \theta\right)$ with noise drawn from $\mathcal{N}(0, E)$, is $I\cdot\frac{\sqrt{2}}{\sqrt{\pi}}$-Lipschitz, where $I$ is the Lipschitz constant of $J\left(x, y, \theta\right)$.

Figures (1)

  • Figure 1: The relationship between the similarity between consecutive gradients computed in attacks and transferability rates. Results obtained using Inc-v3 as a surrogate and the remaining ImageNet models as targets. Gauss-DST refers attacks composing Gaussian noise and DST and uses the same number of augmented images as UltCombGen. Notice how UltCombBase and UltCombGen lead to higher similarity between gradients than most attacks, and how the transferability rates tend to increase as the gradient similarity increases.

Theorems & Definitions (3)

  • theorem 1
  • theorem 2
  • theorem 3