Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Geometric Probe of the Accuracy-Robustness Trade-off: Sharp Boundaries in Symmetry-Breaking Dimensional Expansion

Yu Bai, Zhe Wang, Jiarui Zhang, Dong-Xiao Zhang, Yinjun Gao, Jun-Jie Zhang

TL;DR

These findings provide a concrete geometric explanation for the accuracy-robustness paradox: the optimization landscape deepens the basin of attraction to improve accuracy but inevitably erects steep walls along the auxiliary degrees of freedom, creating a fragile sensitivity to off-manifold perturbations.

Abstract

The trade-off between clean accuracy and adversarial robustness is a pervasive phenomenon in deep learning, yet its geometric origin remains elusive. In this work, we utilize Symmetry-Breaking Dimensional Expansion (SBDE) as a controlled probe to investigate the mechanism underlying this trade-off. SBDE expands input images by inserting constant-valued pixels, which breaks translational symmetry and consistently improves clean accuracy (e.g., from $90.47\%$ to $95.63\%$ on CIFAR-10 with ResNet-18) by reducing parameter degeneracy. However, this accuracy gain comes at the cost of reduced robustness against iterative white-box attacks. By employing a test-time \emph{mask projection} that resets the inserted auxiliary pixels to their training values, we demonstrate that the vulnerability stems almost entirely from the inserted dimensions. The projection effectively neutralizes the attacks and restores robustness, revealing that the model achieves high accuracy by creating \emph{sharp boundaries} (steep loss gradients) specifically along the auxiliary axes. Our findings provide a concrete geometric explanation for the accuracy-robustness paradox: the optimization landscape deepens the basin of attraction to improve accuracy but inevitably erects steep walls along the auxiliary degrees of freedom, creating a fragile sensitivity to off-manifold perturbations.

A Geometric Probe of the Accuracy-Robustness Trade-off: Sharp Boundaries in Symmetry-Breaking Dimensional Expansion

TL;DR

These findings provide a concrete geometric explanation for the accuracy-robustness paradox: the optimization landscape deepens the basin of attraction to improve accuracy but inevitably erects steep walls along the auxiliary degrees of freedom, creating a fragile sensitivity to off-manifold perturbations.

Abstract

The trade-off between clean accuracy and adversarial robustness is a pervasive phenomenon in deep learning, yet its geometric origin remains elusive. In this work, we utilize Symmetry-Breaking Dimensional Expansion (SBDE) as a controlled probe to investigate the mechanism underlying this trade-off. SBDE expands input images by inserting constant-valued pixels, which breaks translational symmetry and consistently improves clean accuracy (e.g., from to on CIFAR-10 with ResNet-18) by reducing parameter degeneracy. However, this accuracy gain comes at the cost of reduced robustness against iterative white-box attacks. By employing a test-time \emph{mask projection} that resets the inserted auxiliary pixels to their training values, we demonstrate that the vulnerability stems almost entirely from the inserted dimensions. The projection effectively neutralizes the attacks and restores robustness, revealing that the model achieves high accuracy by creating \emph{sharp boundaries} (steep loss gradients) specifically along the auxiliary axes. Our findings provide a concrete geometric explanation for the accuracy-robustness paradox: the optimization landscape deepens the basin of attraction to improve accuracy but inevitably erects steep walls along the auxiliary degrees of freedom, creating a fragile sensitivity to off-manifold perturbations.
Paper Structure (15 sections, 5 equations, 3 figures, 5 tables)

This paper contains 15 sections, 5 equations, 3 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Method: a controlled probe of loss geometry
  3. Symmetry-breaking expansion: motivation and operator
  4. Iterative White-Box Attacks as a Local Geometry Probe
  5. Mask Projection for Verification
  6. Experimental Setup
  7. Model and Data
  8. Training
  9. Attacks
  10. Threat Model
  11. Results
  12. SBDE raises clean accuracy yet reduces robustness
  13. Projection $\Pi$ cancels the attack while preserving accuracy
  14. Ablations
  15. Discussion and Conclusion

Figures (3)

  • Figure 1: Symmetry-breaking dimensional expansion (illustration). Original image pixels are mapped to designated locations in the expanded grid; constant-valued rows/columns fill the rest.
  • Figure 2: Test accuracy trajectory on raw vs. SBDE-expanded data (ResNet-18 on CIFAR-10).
  • Figure 3: Perturbation trajectory after an iterative attack. The Z-axis corresponds to the value of the loss function for the attacked image at each iterative step. The X-axis and Y-axis correspond to the pixel value of a randomly selected pixel and its adjacent auxiliary pixel, respectively. Blue lines represent the iterative attacks of PGD (with the increase of the step, the loss value increases). Red line links the start point (no attack) and end point (Projection of attacked image). After projection, the loss function drops prominently, indicating that high-magnitude pixels concentrate on SBDE’s auxiliary coordinates $\Omega_{\text{aux}}$.