Asymptotic Midpoint Mixup for Margin Balancing and Moderate Broadening

TL;DR

This work tackles the collapse of features encountered in interpolation-based augmentations by introducing Asymptotic Midpoint Mixup (AM-mixup). By generating interpolated features that asymptotically approach the midpoint between inter-class centers and labeling them on a single side, AM-mixup balances class margins and moderately broadens them, mitigating both intra-class and inter-class collapse. The approach is evaluated on coarse-to-fine transfer and long-tailed image classification, with analyses of alignment and uniformity showing reduced collapse relative to Mixup and Manifold Mixup. Key contributions include a formalization of intra- and inter-class collapse, a novel AM-mixup mechanism with a decaying mixup parameter, and extensive ablations demonstrating its effectiveness across datasets and backbones, albeit with some task-dependent hyperparameter sensitivity.

Abstract

In the feature space, the collapse between features invokes critical problems in representation learning by remaining the features undistinguished. Interpolation-based augmentation methods such as mixup have shown their effectiveness in relieving the collapse problem between different classes, called inter-class collapse. However, intra-class collapse raised in coarse-to-fine transfer learning has not been discussed in the augmentation approach. To address them, we propose a better feature augmentation method, asymptotic midpoint mixup. The method generates augmented features by interpolation but gradually moves them toward the midpoint of inter-class feature pairs. As a result, the method induces two effects: 1) balancing the margin for all classes and 2) only moderately broadening the margin until it holds maximal confidence. We empirically analyze the collapse effects by measuring alignment and uniformity with visualizing representations. Then, we validate the intra-class collapse effects in coarse-to-fine transfer learning and the inter-class collapse effects in imbalanced learning on long-tailed datasets. In both tasks, our method shows better performance than other augmentation methods.

Figures (5)

• Figure 1: Overview of Asymptotic Midpoint Mixup. (left) Feature vectors of input samples came from the pre-trained encoder. (middle) Asymptotic midpoint mixup generates augmented features and their labels based on interpolation. Examples for understanding the interpolation are highlighted as cyan. The ratio between two different features is controlled by $\lambda$ and this parameter has asymptotically decreased from 1.0 to 0.5 until the end of training. The augmented features are created as a mini-batch size at the same rate. (right) Finally, the augmented features are passed to the classifier.
• Figure 2: Comparison results on Mini-CIFAR-Coarse and Mini-CIFAR-Fine. In the case of coarse-to-fine transfer learning, features have various scales when trained on the coarse-grained datasets, and this diversity of the scale is helpful for fine-tuning on the fine-grained datasets. Our method shows better spread in the feature space than mixup and manifold mixup but not enough to CE. The background shows the confidence landscape as a heatmap, where the lighter the color, the higher the confidence (min: 0.0, max: 1.0). Features are colored according to the class to which they belong. (CE: Cross entropy without any augmentation method, AM-mixup: our method)
• Figure 3: Comparison results on Mini-CIFAR and Mini-CIFAR-LT. In a normal environment, CE shows that the features have an extremely large scale, while the scale of the features in mixup and manifold mixup is very small. In between, our method shows moderately broadening the margins of classes than others. In imbalanced learning, the centroids of the tail classes (bird and cat) are very close to the decision boundary in mixup and manifold mixup. CE shows worse results as classifying input samples in the tail classes to the head class (airplane). The background shows the confidence landscape as a heatmap, where the lighter the color, the higher the confidence (min: 0.0, max: 1.0). Features are colored according to the class to which they belong. (CE: Cross entropy without any augmentation method, AM-mixup: our method)
• Figure 4: Visualization of the results of motivation test. Each alignment is printed on the figures.
• Figure 5: Visualization of the results of motivation test. Each uniformity and neighbor uniformity are printed on the figures.