On the Effects of Adversarial Perturbations on Distribution Robustness

TL;DR

The paper investigates how adversarial perturbations affect distribution robustness under data shifts caused by spurious correlations. It adopts a two-stage proxy framework with Gaussian-feature modeling to derive closed-form expressions for training and test accuracy under clean and perturbed data, revealing how feature separability and spurious correlation severity mediate the adversarial-distribution robustness tradeoff. Surprisingly, under moderately biased data, L_inf perturbations can increase distribution robustness by reducing reliance on spurious features, and the gain can persist on highly skewed data when the core feature is highly separable. The results are validated on synthetic data and show that the surrogate analysis captures key aspects of standard adversarial training, offering guidance on when perturbations help or hurt distribution robustness.

Abstract

Adversarial robustness refers to a model's ability to resist perturbation of inputs, while distribution robustness evaluates the performance of the model under data shifts. Although both aim to ensure reliable performance, prior work has revealed a tradeoff in distribution and adversarial robustness. Specifically, adversarial training might increase reliance on spurious features, which can harm distribution robustness, especially the performance on some underrepresented subgroups. We present a theoretical analysis of adversarial and distribution robustness that provides a tractable surrogate for per-step adversarial training by studying models trained on perturbed data. In addition to the tradeoff, our work further identified a nuanced phenomenon that $\ell_\infty$ perturbations on data with moderate bias can yield an increase in distribution robustness. Moreover, the gain in distribution robustness remains on highly skewed data when simplicity bias induces reliance on the core feature, characterized as greater feature separability. Our theoretical analysis extends the understanding of the tradeoff by highlighting the interplay of the tradeoff and the feature separability. Despite the tradeoff that persists in many cases, overlooking the role of feature separability may lead to misleading conclusions about robustness.

Figures (8)

• Figure 1: Examples of Distribution Robustness and Adversarial Perturbations. Groups are denoted by their two binary attributes. The left subplots show distribution robustness when training data contains spurious correlation ($\zeta=0.95$; 95% majority) and test data is balanced, leading to an accuracy drop under shift. The right subplots show training data perturbed with $\ell_\infty$ at varying feature separabilities and a large attack budget of 1 to highlight the effects. When the spurious feature is easier to classify, as $m_1 < m_2$, the classifier increasingly relies on the spurious feature (feature 2); conversely, $m_1 > m_2$ increases the reliance on the non-spurious feature.
• Figure 2: Theoretical accuracy gap between proxy-trained and clean-trained models under $\ell_2$ perturbations.
• Figure 3: Theoretical accuracy gap between proxy-trained and clean-trained models under $\ell_\infty$ perturbations.
• Figure 4: Empirical accuracy gap between proxy-trained and clean-trained models under $\ell_2$ adversarial perturbations.
• Figure 5: Empirical accuracy gap between proxy-trained and clean-trained models under $\ell_\infty$ adversarial perturbations.

Theorems & Definitions (5)

• proof
• proof
• proof
• proof
• proof