Overfitting in adversarially robust deep learning

TL;DR

The paper reveals that robust overfitting is a widespread phenomenon in adversarially trained networks, where longer training after initial learning-rate decays degrades robust test performance. It demonstrates that early stopping, including validation-based stopping, can match or surpass many algorithmic advances like TRADES on robustness across multiple datasets and threat models. Classical regularization and data augmentation offer limited, often context-dependent gains and rarely exceed the benefits of early stopping, though combining semi-supervised augmentation with early stopping can provide notable improvements. The findings advocate for validation-driven model selection and highlight a fundamental distinction between standard and robust generalization, with practical implications for evaluating and deploying adversarial defenses.

Abstract

It is common practice in deep learning to use overparameterized networks and train for as long as possible; there are numerous studies that show, both theoretically and empirically, that such practices surprisingly do not unduly harm the generalization performance of the classifier. In this paper, we empirically study this phenomenon in the setting of adversarially trained deep networks, which are trained to minimize the loss under worst-case adversarial perturbations. We find that overfitting to the training set does in fact harm robust performance to a very large degree in adversarially robust training across multiple datasets (SVHN, CIFAR-10, CIFAR-100, and ImageNet) and perturbation models ($\ell_\infty$ and $\ell_2$). Based upon this observed effect, we show that the performance gains of virtually all recent algorithmic improvements upon adversarial training can be matched by simply using early stopping. We also show that effects such as the double descent curve do still occur in adversarially trained models, yet fail to explain the observed overfitting. Finally, we study several classical and modern deep learning remedies for overfitting, including regularization and data augmentation, and find that no approach in isolation improves significantly upon the gains achieved by early stopping. All code for reproducing the experiments as well as pretrained model weights and training logs can be found at https://github.com/locuslab/robust_overfitting.

Figures (27)

• Figure 1: The learning curves for a robustly trained model replicating the experiment done by madry2017towards on CIFAR-10. The curves demonstrate "robust overfitting"; shortly after the first learning rate decay the model momentarily attains 43.2% robust error, and is actually more robust than the model at the end of training, which only attains 51.4% robust test error against a 10-step PGD adversary for $\ell_\infty$ radius of $\epsilon=8/255$. The learning rate is decayed at 100 and 150 epochs.
• Figure 2: Robust test error over training epochs for various learning rate schedules on CIFAR-10. None of the alternative smoother learning rate schedules can achieve a peak performance competitive with the standard piecewise decay learning rate, indicating that the peak performance is obtained by having a single discrete jump. Note that the multiple decay schedule is actually run for 500 epochs, but compressed into this plot for a clear comparison.
• Figure 3: Learning curves showing standard and robust error rates for a Wide ResNet model trained with TRADES on CIFAR-10. Early stopping after the initial learning rate decay is crucial in order to achieve the 43.4% robust test error reported by zhang2019theoretically, which eventually degrades to 50.6% robust test error when the training has converged.
• Figure 4: Learning curves for a CIFAR-10 pre-activation ResNet18 model trained with a hold-out validation set of 1,000 examples. We find that the hold-out validation set is enough to reflect the test set performance, and stopping based on the validation set is able to prevent overfitting and recover 46.9% robust test error, in comparison to 46.7% achieved by the best-performing model checkpoint.
• Figure 5: Generalization curves depicting double descent for adversarially robust generalization, where hypothesis class complexity is controlled by varying the width factor for a wide residual network. Each final model point represents the average performance over the last 5 epochs with the corresponding width factor from training until convergence. The best checkpoint refers to the lowest robust test error achieved by a model checkpoint during training, and illustrates the significant gap in performance between the best and final models resulting from robust overfitting.