Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On the Impact of Hard Adversarial Instances on Overfitting in Adversarial Training

Chen Liu, Zhichao Huang, Mathieu Salzmann, Tong Zhang, Sabine Süsstrunk

TL;DR

The paper identifies adversarial overfitting in adversarial training as arising from fitting hard training instances. It introduces a principled instance-difficulty metric and provides rigorous linear- and nonlinear-model analyses showing that harder instances worsen robust generalization, with effects amplified by larger adversarial budgets. Empirically, it demonstrates that mitigating strategies that adapt inputs or targets—including subset selection, fast training, and adversarial fine-tuning with extra data—improve robustness by avoiding or downweighting hard instances, while methods that emphasize hard examples may fail under adaptive threats. The work offers a unified, instance-centric view of robustness and provides practical guidance for designing adaptive training schemes to enhance adversarial robustness.

Abstract

Adversarial training is a popular method to robustify models against adversarial attacks. However, it exhibits much more severe overfitting than training on clean inputs. In this work, we investigate this phenomenon from the perspective of training instances, i.e., training input-target pairs. Based on a quantitative metric measuring the relative difficulty of an instance in the training set, we analyze the model's behavior on training instances of different difficulty levels. This lets us demonstrate that the decay in generalization performance of adversarial training is a result of fitting hard adversarial instances. We theoretically verify our observations for both linear and general nonlinear models, proving that models trained on hard instances have worse generalization performance than ones trained on easy instances, and that this generalization gap increases with the size of the adversarial budget. Finally, we investigate solutions to mitigate adversarial overfitting in several scenarios, including fast adversarial training and fine-tuning a pretrained model with additional data. Our results demonstrate that using training data adaptively improves the model's robustness.

On the Impact of Hard Adversarial Instances on Overfitting in Adversarial Training

TL;DR

The paper identifies adversarial overfitting in adversarial training as arising from fitting hard training instances. It introduces a principled instance-difficulty metric and provides rigorous linear- and nonlinear-model analyses showing that harder instances worsen robust generalization, with effects amplified by larger adversarial budgets. Empirically, it demonstrates that mitigating strategies that adapt inputs or targets—including subset selection, fast training, and adversarial fine-tuning with extra data—improve robustness by avoiding or downweighting hard instances, while methods that emphasize hard examples may fail under adaptive threats. The work offers a unified, instance-centric view of robustness and provides practical guidance for designing adaptive training schemes to enhance adversarial robustness.

Abstract

Adversarial training is a popular method to robustify models against adversarial attacks. However, it exhibits much more severe overfitting than training on clean inputs. In this work, we investigate this phenomenon from the perspective of training instances, i.e., training input-target pairs. Based on a quantitative metric measuring the relative difficulty of an instance in the training set, we analyze the model's behavior on training instances of different difficulty levels. This lets us demonstrate that the decay in generalization performance of adversarial training is a result of fitting hard adversarial instances. We theoretically verify our observations for both linear and general nonlinear models, proving that models trained on hard instances have worse generalization performance than ones trained on easy instances, and that this generalization gap increases with the size of the adversarial budget. Finally, we investigate solutions to mitigate adversarial overfitting in several scenarios, including fast adversarial training and fine-tuning a pretrained model with additional data. Our results demonstrate that using training data adaptively improves the model's robustness.
Paper Structure (31 sections, 12 theorems, 49 equations, 15 figures, 6 tables, 1 algorithm)

This paper contains 31 sections, 12 theorems, 49 equations, 15 figures, 6 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. A Metric for Instance Difficulty
  4. Hard Instances Lead to Overfitting
  5. Using a Subset of Training Data
  6. Using the Whole Training Set
  7. Theoretical Analysis
  8. Linear Models
  9. General Nonlinear Models
  10. Numerical Simulation
  11. Case Study and Discussion
  12. Standard Adversarial Training: A New Perspective on Existing Methods
  13. Alternative Training Scenarios
  14. Fast Adversarial Training
  15. Adversarial Fine-tuning with Additional Data
  16. ...and 16 more sections

Key Result

Theorem 1

For a dataset $\{{\bm{x}}_i, y_i\}_{i = 1}^n$ that is linearly separable under the adversarial budget ${\mathcal{S}}^{(2)}(\epsilon)$, any initial point ${\bm{w}}_0$ and step size $\alpha \leq 2\|{\mathbf{X}}\|^{-2}$, the gradient descent ${\bm{w}}_{u + 1} = {\bm{w}}_u - \alpha \triangledown_{{\bm{w

Figures (15)

  • Figure 1: Some examples of the easiest and the hardest instances in CIFAR10 krizhevsky2009learning and SVHN netzer2011reading datasets. We pick some examples from the "plane" category in CIFAR10 and "0" category in SVHN. The number on top of each image indicates the corresponding value of the difficulty function
  • Figure 2: Learning curves obtained by training on the $10000$ easiest, random and hardest instances of CIFAR10 under different scenarios. The training error (dashed lines) is the error on the selected instances, and the test error (solid lines) is the error on the whole test set. The y-axis of each subfigure indicates the training or test error under the corresponding perturbation, so the error rates of different subfigures are not comparable.
  • Figure 3: (a) The training error (dashed line) and the test error (solid line) when we conduct adversarial training on the $10000$ hardest training instances for more epochs until convergence. (b) The learning curves of training on the $10000$ hardest training instances when we use a different optimizer, including different learning rates and a different algorithm. (c) The learning curves on the training (dash lines) and the test (solid lines) sets when we remove the hardest training instances.
  • Figure 4: Analysis on the groups ${\mathcal{G}}_0$, ${\mathcal{G}}_3$, ${\mathcal{G}}_6$ and ${\mathcal{G}}_9$ in the training set. The right vertical axis corresponds to the training (dashed grey line) and test (solid grey line) error under adversarial attacks for both plots. Left plot: The left vertical axis represents the average loss of different groups. Right plot: The left vertical axis represents the average $l_2$ norm of features extracted during training for different groups.
  • Figure 5: Curves of the Lipschitz upper bound when the model is adversarially trained by the easiest, random, the hardest 10000 instances or the whole training set. The y-axis is in log-scale. Left: the adversarial budget is based on the $l_\infty$ norm with $\epsilon = 8 / 255$. Right: the adversarial budget is based on the $l_2$ norm with $\epsilon = 1$.
  • ...and 10 more figures

Theorems & Definitions (13)

  • Theorem 1
  • Theorem 2
  • Corollary 3
  • Definition 5
  • Lemma 6
  • Theorem 7
  • Corollary 8
  • Lemma 9
  • Lemma 10
  • Lemma 11
  • ...and 3 more