Boosting Model Resilience via Implicit Adversarial Data Augmentation

TL;DR

The paper tackles learning under data biases such as long-tail, noisy labels, and subpopulation shifts by introducing Implicit Adversarial Data Augmentation (IADA), which augments samples in the deep feature space via per-sample adversarial and anti-adversarial perturbations. A surrogate loss, $ ext{L}^{IADA}$, is derived from the infinite-augmentation limit, and a meta-learning framework, Meta-IADA, trains a perturbation network to produce per-sample perturbations $epsilon_i$ guiding classifier optimization. Key contributions include the IADA loss, a per-sample perturbation mechanism, and extensive empirical validation across LT, GLT, noisy-label, and subpopulation-shift benchmarks showing state-of-the-art performance and improved robustness and fairness. The approach promises practical impact by enabling robust generalization without explicit augmentation, adapting to diverse biases through sample-wise perturbation strategies guided by a small meta-dataset.

Abstract

Data augmentation plays a pivotal role in enhancing and diversifying training data. Nonetheless, consistently improving model performance in varied learning scenarios, especially those with inherent data biases, remains challenging. To address this, we propose to augment the deep features of samples by incorporating their adversarial and anti-adversarial perturbation distributions, enabling adaptive adjustment in the learning difficulty tailored to each sample's specific characteristics. We then theoretically reveal that our augmentation process approximates the optimization of a surrogate loss function as the number of augmented copies increases indefinitely. This insight leads us to develop a meta-learning-based framework for optimizing classifiers with this novel loss, introducing the effects of augmentation while bypassing the explicit augmentation process. We conduct extensive experiments across four common biased learning scenarios: long-tail learning, generalized long-tail learning, noisy label learning, and subpopulation shift learning. The empirical results demonstrate that our method consistently achieves state-of-the-art performance, highlighting its broad adaptability.

Figures (6)

• Figure 1: (a) Illustration for our augmentation strategy, which augments samples within their adversarial and anti-adversarial perturbation distributions. (b) Illustration of an imbalanced learning scenario. Our method employs adversarial and anti-adversarial augmentations for the minor and major classes, respectively.
• Figure 2: The overview of our method pipeline. We initiate with a sample-wise adversarial data augmentation strategy (Box 1), enriching the deep features of samples using perturbation vectors extracted from their adversarial and anti-adversarial perturbation distributions. Subsequently, by considering an infinite number of augmented instances, we derive a novel robust loss, termed IADA (Box 2). Regularization analysis reveals the efficacy of IADA in improving model generalization, robustness, and inter-class fairness. To facilitate optimization with IADA, we then establish a meta-learning-based framework called Meta-IADA (Box 3). Within it, a perturbation network is tasked with generating perturbation strategies for samples (denoted as $\epsilon_{\boldsymbol{x}}$) in the IADA loss, leveraging a set of ($K\!=\!15$) training characteristics as inputs.
• Figure 3: Ratio of adversarial samples in each class during the last forty epochs on CIFAR10 under imbalance ratios of 10:1 and 100:1. From "C1" to "C10", the class proportions progressively decrease.
• Figure 4: Ratio of adversarial samples for noisy and clean samples in the final forty epochs on CIFAR10 with 20% and 40% flip noise.
• Figure 5: Visualization of instances corresponding to deep features augmented by Meta-IADA.