IPMix: Label-Preserving Data Augmentation Method for Training Robust Classifiers

TL;DR

IPMix addresses the need for robust classifiers under distribution shifts without sacrificing clean accuracy. It unifies image-level, patch-level, and pixel-level augmentations into a label-preserving framework that also injects structural complexity via fractal synthetic data and random, multi-scale information fusion. The method uses a simple, non-search-based mixing scheme with Dirichlet and Beta weightings, formalized as $\tilde{x} = B \odot x_1 + (I - B) \odot x_2$ and $x_{IPMix} = m \cdot x_{mix} + (1 - m) \cdot x$, enabling effective information fusion. Experiments on CIFAR-10/100 and ImageNet show IPMix achieves state-of-the-art or comparable results across corruption robustness, calibration, adversarial robustness, anomaly detection, and prediction consistency, with modest training overhead and strong qualitative indications of improved feature diversity and boundary coverage.

Abstract

Data augmentation has been proven effective for training high-accuracy convolutional neural network classifiers by preventing overfitting. However, building deep neural networks in real-world scenarios requires not only high accuracy on clean data but also robustness when data distributions shift. While prior methods have proposed that there is a trade-off between accuracy and robustness, we propose IPMix, a simple data augmentation approach to improve robustness without hurting clean accuracy. IPMix integrates three levels of data augmentation (image-level, patch-level, and pixel-level) into a coherent and label-preserving technique to increase the diversity of training data with limited computational overhead. To further improve the robustness, IPMix introduces structural complexity at different levels to generate more diverse images and adopts the random mixing method for multi-scale information fusion. Experiments demonstrate that IPMix outperforms state-of-the-art corruption robustness on CIFAR-C and ImageNet-C. In addition, we show that IPMix also significantly improves the other safety measures, including robustness to adversarial perturbations, calibration, prediction consistency, and anomaly detection, achieving state-of-the-art or comparable results on several benchmarks, including ImageNet-R, ImageNet-A, and ImageNet-O.

Figures (9)

• Figure 1: Visual comparison of various data augmentation methods. IPMix utilizes the structural complexity of fractals and multi-scale information to generate more diverse examples.
• Figure 2: The performance of different levels of data augmentation methods on CIFAR-100. Compared to other approaches which focus on utilizing only one category, IPMix achieves state-of-the-art accuracy and robustness.
• Figure 3: Top: Sample fractals from IPMix set. Bottom: An example of IPMix applied on a dog image, $k$ = 2, $t$ = 3. We randomly select P (pixel and patch) data augmentation methods and image-level data augmentation methods to generate a highly diverse set of augmented images. We sample $w_k$ ($k$ = 2, in this case) from Dirichlet distribution and use skip connection ($m$ sample from a Beta distribution) to maintain semantic consistency.
• Figure 4: Different mixing framework of IPMix. P augmentation operation represents pixel-level and patch-level augmentation operations. ① Utilizing P operations and image-level operations in different chains and mixing the results. ② A clean image is randomly carried out by P operations or image-level operations in linear combinations to generate an IPMix image. ③ leveraging the mixed image as a new input.
• Figure 5: Top: Examples of random mixing operations. Bottom: Examples of IPMix-Scar mixing and IPMix-Square mixing.