AggSS: An Aggregated Self-Supervised Approach for Class-Incremental Learning

TL;DR

Class-incremental learning often suffers catastrophic forgetting; AggSS mitigates this by leveraging rotational self-supervision as auxiliary classes. By treating each rotation as a separate class (with $M=4$) and aggregating the rotation-specific logits at inference, AggSS strengthens feature representations and provides a plug-in augmentation for existing CIL frameworks, via a dedicated loss ${\\mathcal{L}_{AggSS}}$ and rotated-ensemble inference ${p^{agg}}$. Empirical results on CIFAR-100 and ImageNet-Subset show substantial accuracy gains across traditional, long-tail, and semi-supervised CIL settings, supported by qualitative Grad-CAM analyses that reveal attention shifts to intrinsic object features. Overall, AggSS offers a practical, scalable approach to enhance continual learning systems by enriching representations through self-supervised rotations.

Abstract

This paper investigates the impact of self-supervised learning, specifically image rotations, on various class-incremental learning paradigms. Here, each image with a predefined rotation is considered as a new class for training. At inference, all image rotation predictions are aggregated for the final prediction, a strategy we term Aggregated Self-Supervision (AggSS). We observe a shift in the deep neural network's attention towards intrinsic object features as it learns through AggSS strategy. This learning approach significantly enhances class-incremental learning by promoting robust feature learning. AggSS serves as a plug-and-play module that can be seamlessly incorporated into any class-incremental learning framework, leveraging its powerful feature learning capabilities to enhance performance across various class-incremental learning approaches. Extensive experiments conducted on standard incremental learning datasets CIFAR-100 and ImageNet-Subset demonstrate the significant role of AggSS in improving performance within these paradigms.

Figures (9)

• Figure 1: Illustrates various scenarios of class-incremental learning based on the available data characteristics at each incremental task: (i) Traditional CIL, where an abundant amount of labeled data is present at every task; (ii) Long-Tail CIL, where data follows long-tail distributions at every task; (iii) Few-shot CIL, where each class contains very few samples at each task; (iv) Semi-supervised CIL, where the model has access to both labeled and unlabeled data at each task; and (v) Unsupervised CIL, where every task has access only to unlabeled data.
• Figure 2: Illustrates both training and testing strategy of AggSS.
• Figure 3: Experiment results on semi-supervised CIL setting.
• Figure 4: Performance vs Rotations
• Figure 5: Displays GradCAM patterns for images trained using both conventional cross-entropy (CE) loss and AggSS loss. In AggSS, each rotation is treated as a separate class.