Improving the Accuracy-Robustness Trade-Off of Classifiers via Adaptive Smoothing

TL;DR

This work tackles the persistent accuracy-robustness trade-off in neural classifiers by introducing adaptive smoothing, a framework that blends outputs from a high-accuracy standard model $g$ and a robust model $h$ via a mixing mechanism. It first replaces the smoothing oracle with a robust neural network, enabling a scalable multi-class smoothing and providing theoretical certificates for the mixed classifier’s robustness under both Lipschitz and randomized smoothing assumptions. It then augments the framework with a mixing network that adapts the strength of smoothing per input, trained with a composite loss to balance clean accuracy and adversarial robustness, and evaluated against strong attacks including AutoAttack. Empirically, adaptive smoothing yields substantial clean accuracy gains while maintaining near-SOTA robustness, achieving state-of-the-art trade-offs on CIFAR-100 and competitive results on CIFAR-10, with flexible compatibility to future robustness improvements and adversary detection techniques. The combination of theoretical guarantees and practical performance demonstrates that the accuracy-robustness gap can be closed without retraining robust models for every desired balance, enabling scalable deployment of robust systems.

Abstract

While prior research has proposed a plethora of methods that build neural classifiers robust against adversarial robustness, practitioners are still reluctant to adopt them due to their unacceptably severe clean accuracy penalties. This paper significantly alleviates this accuracy-robustness trade-off by mixing the output probabilities of a standard classifier and a robust classifier, where the standard network is optimized for clean accuracy and is not robust in general. We show that the robust base classifier's confidence difference for correct and incorrect examples is the key to this improvement. In addition to providing intuitions and empirical evidence, we theoretically certify the robustness of the mixed classifier under realistic assumptions. Furthermore, we adapt an adversarial input detector into a mixing network that adaptively adjusts the mixture of the two base models, further reducing the accuracy penalty of achieving robustness. The proposed flexible method, termed "adaptive smoothing", can work in conjunction with existing or even future methods that improve clean accuracy, robustness, or adversary detection. Our empirical evaluation considers strong attack methods, including AutoAttack and adaptive attack. On the CIFAR-100 dataset, our method achieves an 85.21% clean accuracy while maintaining a 38.72% $\ell_\infty$-AutoAttacked ($ε= 8/255$) accuracy, becoming the second most robust method on the RobustBench CIFAR-100 benchmark as of submission, while improving the clean accuracy by ten percentage points compared with all listed models. The code that implements our method is available at https://github.com/Bai-YT/AdaptiveSmoothing.

Figures (9)

• Figure 1: Compare the "attacked accuracy -- clean accuracy" curves for various $R_i (x)$ options.
• Figure 1: The overall architecture of the adaptively smoothed classifier introduced in \ref{['sec:ada_smo']}. "RNB" stands for ResNetBlock and "BN" represents the 2D batch normalization layer.
• Figure 1: The performance of the mixed classifier $h^{\alpha} (\cdot)$. "STD attack", "ROB attack", and "MIX attack" refer to the PGD$_{20}$ attack generated using the gradient of $g (\cdot)$, $h (\cdot)$, and $h^{\alpha} (\cdot)$ respectively, with $\epsilon$ set to $8/255$.
• Figure 1: Closed-form certified accuracy of RS models and our mixed classifier with the Lipschitz-based bound in \ref{['thm:certified_radius']}. The mixed classifier can optimize the certified robust accuracy at each radius without affecting clean accuracy by tuning $\alpha$ and $\sigma^2$. The resulting Pareto frontier demonstrates significantly extended certified radii over a standalone RS model, signaling improved accuracy-robustness trade-off.
• Figure 1: Comparing the "attacked accuracy versus clean accuracy" curve of various options for $R_i (x)$ with alternative selections of base classifiers.

Theorems & Definitions (10)

• Definition 3.1
• Lemma 3.2
• Proof 1
• Definition 3.3
• Theorem 3.5
• Proof 2
• Theorem 4.1
• Proof 3
• Theorem A.2
• Proof 4