Catastrophic Overfitting: A Potential Blessing in Disguise

TL;DR

Catastrophic overfitting (CO) arises in fast adversarial training (FAT) when adversarial accuracy collapses despite high clean accuracy. The authors diagnose CO by analyzing feature activation differences $V_{act}$ across five activation nodes in ResNet18 on CIFAR-10 and show CO localizes to a small subset of channels with large saliency quantified by $T_{act}^{i,k}$. They introduce two regularizers, $\\mathcal{L}_{stable}$ and $\\mathcal{L}_{co}$, to mitigate or induce CO by dampening or amplifying activation differences on selected channels, with minimal hyperparameter sensitivity under stable training. Building on CO, they demonstrate attack obfuscation by evaluating CO-affected models with random input noise $\\delta_R$, achieving optimal accuracy on both clean and adversarial data and robustness to transferred attacks; this challenges the view that CO must be eliminated. Overall, the work reframes CO from a defect to a potential asset for improving FAT robustness via targeted pathway manipulation and noisy inference.

Abstract

Fast Adversarial Training (FAT) has gained increasing attention within the research community owing to its efficacy in improving adversarial robustness. Particularly noteworthy is the challenge posed by catastrophic overfitting (CO) in this field. Although existing FAT approaches have made strides in mitigating CO, the ascent of adversarial robustness occurs with a non-negligible decline in classification accuracy on clean samples. To tackle this issue, we initially employ the feature activation differences between clean and adversarial examples to analyze the underlying causes of CO. Intriguingly, our findings reveal that CO can be attributed to the feature coverage induced by a few specific pathways. By intentionally manipulating feature activation differences in these pathways with well-designed regularization terms, we can effectively mitigate and induce CO, providing further evidence for this observation. Notably, models trained stably with these terms exhibit superior performance compared to prior FAT work. On this basis, we harness CO to achieve `attack obfuscation', aiming to bolster model performance. Consequently, the models suffering from CO can attain optimal classification accuracy on both clean and adversarial data when adding random noise to inputs during evaluation. We also validate their robustness against transferred adversarial examples and the necessity of inducing CO to improve robustness. Hence, CO may not be a problem that has to be solved.

Figures (12)

• Figure 1: Relationship between catastrophic overfitting and feature activation differences $V_{act}$ during adversarial training. We select CIFAR-10 krizhevsky2009learning as the dataset and ResNet18 he2016deep as the network $\mathcal{M}$. Five activation nodes $\mathcal{M}_{\text{A}\sim\text{E}}$ are chosen from $\mathcal{M}$. Left y-axis: Model robustness against the PGD-10 attack; Right y-axis: $V_{act}$ on various activation nodes. $V_{act}$ is quantified using $\mathcal{L}_2$ regularization between features of clean and adversarial examples.
• Figure 2: Overall architecture and the placement of activation nodes. We select five activation nodes in the network ResNet18, and each node locates after a ReLU function.
• Figure 3: Statistical analyses of channel-specific feature activation differences at the activation node $\mathcal{M}_\text{A}$.
• Figure 4: Classification accuracy of models suffering from CO on clean and adversarial examples under various channel masking thresholds. '$\alpha_2$=1.0' indicates the original classification accuracy. '$\$$High variance' signifies using he2023investigating to mask an equivalent number of channels as our method when '$\alpha_2$=0.5'.
• Figure 5: Investigate whether $\mathcal{L}_{co}$ in Eq. (\ref{['eq4']}) can induce CO.