On the Robustness of Adversarial Training Against Uncertainty Attacks

TL;DR

This work investigates the robustness of uncertainty quantification to adversarial perturbations and demonstrates that adversarial training yields more trustworthy uncertainty estimates without ad-hoc defenses. By formulating a modular framework for uncertainty attacks and introducing the Uncertainty Span, the authors derive theoretical insights showing that AT tends to produce under-confident predictions, which enhances resilience to uncertainty attacks. Empirical validation on 23 RobustBench models across CIFAR-10 and ImageNet confirms these trends, with MUS, MSUS, and $s$-ECE providing effective measures of robustness, and OOD/OSR experiments showing practical gains. The study highlights the potential of levering existing robust models to secure downstream uncertainty-based decision making in security-sensitive applications, while noting limitations and directions for extending the analysis to epistemic uncertainty and Bayesian frameworks.

Abstract

In learning problems, the noise inherent to the task at hand hinders the possibility to infer without a certain degree of uncertainty. Quantifying this uncertainty, regardless of its wide use, assumes high relevance for security-sensitive applications. Within these scenarios, it becomes fundamental to guarantee good (i.e., trustworthy) uncertainty measures, which downstream modules can securely employ to drive the final decision-making process. However, an attacker may be interested in forcing the system to produce either (i) highly uncertain outputs jeopardizing the system's availability or (ii) low uncertainty estimates, making the system accept uncertain samples that would instead require a careful inspection (e.g., human intervention). Therefore, it becomes fundamental to understand how to obtain robust uncertainty estimates against these kinds of attacks. In this work, we reveal both empirically and theoretically that defending against adversarial examples, i.e., carefully perturbed samples that cause misclassification, additionally guarantees a more secure, trustworthy uncertainty estimate under common attack scenarios without the need for an ad-hoc defense strategy. To support our claims, we evaluate multiple adversarial-robust models from the publicly available benchmark RobustBench on the CIFAR-10 and ImageNet datasets.

Figures (5)

• Figure 1: An illustration of the effect of standard (left) and adversarial (right) training on the Uncertainty Span (i.e., the range of uncertainty values one can obtain by perturbing a sample with a given budget $\epsilon$) for a classification problem visualized through a standard 3-dimensional simplex. While for standard training, the predicted probability converges on the ground truth probability , for adversarial training, it results in less confidence, i.e., has higher entropy than the ground truth probability. As a result, adversarial training reduces the uncertainty span, whose lower and upper bounds are, respectively, an over-confidence and under-confidence attack. For further details, see \ref{['sec:method']} and \ref{['sec:theory']}.
• Figure 2: Entropy distribution before (green on the left) and after the over- (orange on the right) and under-confidence (blue on the right) attacks for all 15 CIFAR-10 robust classifiers, along with a line marking the mean of each distribution.
• Figure 3: Entropy distribution before (green on the left) and after the over- (orange on the right) and under-confidence (blue on the right) attacks for all 8 ImageNet robust classifiers, along with a line marking the mean of each distribution.
• Figure 4: Calibration curves (above) and confidence histograms (below) for each CIFAR-10 model before (green) and after the over- (orange) and under-confidence (blue) attacks.
• Figure 5: Calibration curves (above) and confidence histograms (below) for each ImageNet model before (green) and after the over- (orange) and under-confidence (blue) attacks.

Theorems & Definitions (6)

• Lemma 4.1
• proof
• Lemma 4.2
• proof
• Theorem 4.1
• proof