Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Deep Adversarial Defense Against Multilevel-Lp Attacks

Ren Wang, Yuxuan Li, Alfred Hero

TL;DR

This work tackles the vulnerability of deep networks to adversarial perturbations and the limitation of single-$\ell_p$ defenses. It introduces Efficient Robust Mode Connectivity (ERMC), which links two endpoint models trained for $p=\infty$ and $p=1$ via a Quadratic Bezier path $\phi_{\boldsymbol\theta}(t)$ and optimizes a worst-case objective across $\{1,\infty\}$ using a MSD-based solver, followed by ensemble selection along the path. The method yields a computationally efficient multilevel $\ell_p$ defense and demonstrates superior robustness against $\ell_\infty$, $\ell_2$, and $\ell_1$ attacks compared to AT-$\ell_\infty$, E-AT, and MSD on CIFAR-10/100 with architectures including PreResNet110, WideResNet-28-10, and ViT-base; an ensemble (ERMC-5) further boosts performance while reducing training time by about 36%. Overall, ERMC provides a practical route to universal robustness by integrating targeted adversarial training with mode connectivity and model ensembling, enabling robust deployment across diverse perturbation regimes.

Abstract

Deep learning models have shown considerable vulnerability to adversarial attacks, particularly as attacker strategies become more sophisticated. While traditional adversarial training (AT) techniques offer some resilience, they often focus on defending against a single type of attack, e.g., the $\ell_\infty$-norm attack, which can fail for other types. This paper introduces a computationally efficient multilevel $\ell_p$ defense, called the Efficient Robust Mode Connectivity (EMRC) method, which aims to enhance a deep learning model's resilience against multiple $\ell_p$-norm attacks. Similar to analytical continuation approaches used in continuous optimization, the method blends two $p$-specific adversarially optimal models, the $\ell_1$- and $\ell_\infty$-norm AT solutions, to provide good adversarial robustness for a range of $p$. We present experiments demonstrating that our approach performs better on various attacks as compared to AT-$\ell_\infty$, E-AT, and MSD, for datasets/architectures including: CIFAR-10, CIFAR-100 / PreResNet110, WideResNet, ViT-Base.

Deep Adversarial Defense Against Multilevel-Lp Attacks

TL;DR

This work tackles the vulnerability of deep networks to adversarial perturbations and the limitation of single- defenses. It introduces Efficient Robust Mode Connectivity (ERMC), which links two endpoint models trained for and via a Quadratic Bezier path and optimizes a worst-case objective across using a MSD-based solver, followed by ensemble selection along the path. The method yields a computationally efficient multilevel defense and demonstrates superior robustness against , , and attacks compared to AT-, E-AT, and MSD on CIFAR-10/100 with architectures including PreResNet110, WideResNet-28-10, and ViT-base; an ensemble (ERMC-5) further boosts performance while reducing training time by about 36%. Overall, ERMC provides a practical route to universal robustness by integrating targeted adversarial training with mode connectivity and model ensembling, enabling robust deployment across diverse perturbation regimes.

Abstract

Deep learning models have shown considerable vulnerability to adversarial attacks, particularly as attacker strategies become more sophisticated. While traditional adversarial training (AT) techniques offer some resilience, they often focus on defending against a single type of attack, e.g., the -norm attack, which can fail for other types. This paper introduces a computationally efficient multilevel defense, called the Efficient Robust Mode Connectivity (EMRC) method, which aims to enhance a deep learning model's resilience against multiple -norm attacks. Similar to analytical continuation approaches used in continuous optimization, the method blends two -specific adversarially optimal models, the - and -norm AT solutions, to provide good adversarial robustness for a range of . We present experiments demonstrating that our approach performs better on various attacks as compared to AT-, E-AT, and MSD, for datasets/architectures including: CIFAR-10, CIFAR-100 / PreResNet110, WideResNet, ViT-Base.
Paper Structure (10 sections, 1 theorem, 6 equations, 1 figure, 1 table, 1 algorithm)

This paper contains 10 sections, 1 theorem, 6 equations, 1 figure, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. Adversarial Attacks And Defenses
  4. Robustness Towards Multiple $\ell_p$ Norm Perturbations
  5. Multilevel $\ell_p$-defense
  6. Proposed Methods
  7. Incorporating robustness into mode connectivity
  8. ERMC with model ensemble
  9. Experiments
  10. Conclusion

Key Result

Theorem 1

croce2019provable Suppose that the classifier is piecewise affine. Let $C$ be the convex hull of the union ball of the $\ell_1$ and $\ell_\infty$. If $d\ge 2$ and $\epsilon_1 \in (\epsilon_\infty, d\epsilon_\infty)$, then where $\beta=\frac{\boldsymbol \epsilon_1}{\boldsymbol \epsilon_\infty} - \lfloor \frac{\boldsymbol \epsilon_1}{\boldsymbol \epsilon_\infty}\rfloor$ and $\frac{1}{p}+\frac{1}{q}

Figures (1)

  • Figure 1: ERMC can find paths with high robustness against $\ell_\infty/\ell_2/\ell_1$ attacks by connecting a $\ell_\infty$ model and a $\ell_1$ model. The effectiveness of ERMC is validated on different datasets and model architectures. Upper panels: the accuracy of the clean test and the robust accuracies under $\ell_\infty/\ell_2/\ell_1$-PGD attacks. Lower panels: the associated loss values of clean test data and perturbed test data. (a) and (b): results obtained from the CIFAR-10 and CIFAR-100 datasets, using the PreResNet110 model architecture. (c) and (d): results from the CIFAR-10 dataset, utilizing the WideResNet-28-10 and ViT-base model architectures.

Theorems & Definitions (1)

  • Theorem 1