SynerMix: Synergistic Mixup Solution for Enhanced Intra-Class Cohesion and Inter-Class Separability in Image Classification

TL;DR

SynerMix tackles two key gaps in MixUp-based data augmentation for image classification: the neglect of intra-class mixup and the lack of explicit intra-class cohesion in learned feature spaces. It introduces SynerMix-Intra, which synthesizes a single per-class feature from unaugmented originals to strengthen intra-class cohesion, and then integrates this with an inter-class mixup component (e.g., MixUp or Manifold MixUp) to form SynerMix, with total loss $ ext{L}_{ ext{total}} = eta ext{L}_{ ext{intra}} + (1-eta) ext{L}_{ ext{inter}}$. Empirical results across six datasets show that intra-class mixup yields 0.22–3.09% gains on its own (avg ≈1.25%), while the full SynerMix improves performance over the best of either component by 0.1–3.43% (avg 1.16%) and over the top of the individual intra/inter methods by 0.12–5.16% (avg 1.11%). The method is model-agnostic and extends to other domains (e.g., speech, text), with findings also highlighting accelerated convergence and a nuanced role for gradient stochasticity due to synthesized features. Code is publicly available, enabling adoption and further exploration of synergistic intra- and inter-class mixup strategies.

Abstract

To address the issues of MixUp and its variants (e.g., Manifold MixUp) in image classification tasks-namely, their neglect of mixing within the same class (intra-class mixup) and their inadequacy in enhancing intra-class cohesion through their mixing operations-we propose a novel mixup method named SynerMix-Intra and, building upon this, introduce a synergistic mixup solution named SynerMix. SynerMix-Intra specifically targets intra-class mixup to bolster intra-class cohesion, a feature not addressed by current mixup methods. For each mini-batch, it leverages feature representations of unaugmented original images from each class to generate a synthesized feature representation through random linear interpolation. All synthesized representations are then fed into the classification and loss layers to calculate an average classification loss that significantly enhances intra-class cohesion. Furthermore, SynerMix combines SynerMix-Intra with an existing mixup approach (e.g., MixUp, Manifold MixUp), which primarily focuses on inter-class mixup and has the benefit of enhancing inter-class separability. In doing so, it integrates both inter- and intra-class mixup in a balanced way while concurrently improving intra-class cohesion and inter-class separability. Experimental results on six datasets show that SynerMix achieves a 0.1% to 3.43% higher accuracy than the best of either MixUp or SynerMix-Intra alone, averaging a 1.16% gain. It also surpasses the top-performer of either Manifold MixUp or SynerMix-Intra by 0.12% to 5.16%, with an average gain of 1.11%. Given that SynerMix is model-agnostic, it holds significant potential for application in other domains where mixup methods have shown promise, such as speech and text classification. Our code is publicly available at: https://github.com/wxitxy/synermix.git.

Figures (12)

• Figure 1: Probability of occurrence for intra-class mixup implemented in existing mixup methods such as MixUp and Manifold MixUp.
• Figure 2: t-SNE visualization of feature representations for testing images from the CIFAR-100 dataset: comparing three methods – the baseline (no mixing operation), Manifold MixUp, and SynerMix-Intra.
• Figure 3: Pipeline of loss calculation for SynerMix. It consists of four components including the supplementation component, intra-class mixup component, inter-class mixup component and integration component. $I_1$ through $I_4$ denote the four unaugmented original images, and $A_1$ through $A_4$ represent the augmented images obtained from $I_1$ through $I_4$ by applying a series of augmentation operations such as cropping, flipping, and cutout. Blue and orange colors are used to differentiate the two classes. Notably, many established mixup methods, such as MixUp and Manifold MixUp, can be used to implement the inter-class mixup component. Here, we only present the loss calculation process of MixUp for clarity.
• Figure 4: Toy examples of feature representation synthesis: (a) Illustration of enhancing the cohesion of image feature representations within the same class; (b) Illustration of using feature representations from unaugmented original images for synthesis.
• Figure 5: Comparison of classification accuracies for wo-RA&ER, w-RA, w-ER(M), w-ER(MM), w-RA&ER(M), and w-RA&ER(MM) across datasets and mini-batch sizes.