The Uncanny Valley: Exploring Adversarial Robustness from a Flatness Perspective

TL;DR

The paper investigates how loss-surface flatness with respect to parameters relates to adversarial robustness, revealing an uncanny valley where iterative attacks first sharpen the loss surface and then drive it into flat regions around adversarial examples. It introduces a relative-flatness metric $\kappa^{\phi}_{Tr}(w)$ based on the Hessian trace and derives a third-derivative bound that links flatness, Lipschitz smoothness, and robustness. Empirically, the uncanny valley appears across CNNs and datasets and to a lesser extent in LLM jailbreak scenarios, and adversarial training shifts the valley but does not eliminate it. The work suggests that vanishing curvature alone cannot guarantee robustness; instead, it provides a bound-based framework that couples local flatness with global smoothness to characterize and improve adversarial resilience.

Abstract

Flatness of the loss surface not only correlates positively with generalization, but is also related to adversarial robustness since perturbations of inputs relate non-linearly to perturbations of weights. In this paper, we empirically analyze the relation between adversarial examples and relative flatness with respect to the parameters of one layer. We observe a peculiar property of adversarial examples in the context of relative flatness: during an iterative first-order white-box attack, the flatness of the loss surface measured around the adversarial example first becomes sharper until the label is flipped, but if we keep the attack running, it runs into a flat uncanny valley where the label remains flipped. In extensive experiments, we observe this phenomenon across various model architectures and datasets, even for adversarially trained models. Our results also extend to large language models (LLMs), but due to the discrete nature of the input space and comparatively weak attacks, adversarial examples rarely reach truly flat regions. Most importantly, this phenomenon shows that flatness alone cannot explain adversarial robustness unless we can also guarantee the behavior of the function around the examples. We, therefore theoretically connect relative flatness to adversarial robustness by bounding the third derivative of the loss surface, underlining the need for flatness in combination with a low global Lipschitz constant for a robust model.

Figures (9)

• Figure 1: The Uncanny Valley. During a multi-step adversarial attack, sharpness first increases; then decreases to almost zero (top), while the loss steadily increases (bottom).
• Figure 2: We report the normalized relative sharpness on the attack trajectory for WideResNet-28-4, ResNet-18, VGG11 and DenseNet121 on the test set of CIFAR-10 & CIFAR-100. We observe that adversarial examples first reach a sharp region, and as the attack progress they land in a flat region. We also display the standard deviation of the values on individual inputs.
• Figure 3: We show how far adversarial examples move in image/feature space from the initial image during a PGD-attack; we measure distance with $L_1$, $L_2$, $L_{\infty}$ and cosine dissimilarity i.e. 1 - cosine similarity. We used CIFAR-10, WideResNet-28-4, and PGD with 10 iterations and $\delta$=8/255.
• Figure 4: Here, we evaluate adversarially trained WideResNet-28-4 on CIFAR-10 with varying $\delta$. We attack the resulting models using PGD-$l_\infty$ with $\delta=12/255$, $\text{steps}=20$, shown in Figure (a) & (b), and $\delta=24/255$, $\text{steps}=50$, depicted in Figure (c) & (d).We can see that even for adversarially trained models, we can find uncanny valleys by using a stronger attack.
• Figure 5: Here, we evaluate adversarially trained WideResNet-28-4 on CIFAR-100 with varying $\delta$. We attack the resulting models using PGD-$l_\infty$ with $\delta=12/255$, $\text{steps}=20$, shown in Figure (a) & (b), and $\delta=24/255$, $\text{steps}=50$, depicted in Figure (c) & (d).We can see that even for adversarially trained models, we can find uncanny valleys by using a stronger attack.

Theorems & Definitions (15)

• Definition 1: szegedy2014intriguingpapernot2016limitations and carlini2017towards
• Definition 2
• Definition 3
• Lemma 3
• Proposition 4
• Corollary 4
• Lemma 4
• proof
• Proposition 4
• proof