When Flatness Does (Not) Guarantee Adversarial Robustness
Nils Philipp Walter, Linara Adilova, Jilles Vreeken, Michael Kamp
TL;DR
The paper formalizes how relative flatness in the penultimate layer relates to adversarial robustness, showing flatness yields local, but not global, guarantees. By deriving a closed-form expression for the penultimate-layer Hessian and bounding loss changes via a loss-based adversarial framework, it links curvature, model confidence, and input perturbations. Empirical findings validate the theory and reveal the Uncanny Valley: adversarial examples often lie in large, flat, high-confidence regions, challenging the notion that flat minima imply robust correctness. The work suggests that achieving true global robustness requires notions of flatness that decouple feature-space stability from final-layer confidence and aligns robustness analysis with input-space geometry and model confidence.
Abstract
Despite their empirical success, neural networks remain vulnerable to small, adversarial perturbations. A longstanding hypothesis suggests that flat minima, regions of low curvature in the loss landscape, offer increased robustness. While intuitive, this connection has remained largely informal and incomplete. By rigorously formalizing the relationship, we show this intuition is only partially correct: flatness implies local but not global adversarial robustness. To arrive at this result, we first derive a closed-form expression for relative flatness in the penultimate layer, and then show we can use this to constrain the variation of the loss in input space. This allows us to formally analyze the adversarial robustness of the entire network. We then show that to maintain robustness beyond a local neighborhood, the loss needs to curve sharply away from the data manifold. We validate our theoretical predictions empirically across architectures and datasets, uncovering the geometric structure that governs adversarial vulnerability, and linking flatness to model confidence: adversarial examples often lie in large, flat regions where the model is confidently wrong. Our results challenge simplified views of flatness and provide a nuanced understanding of its role in robustness.