Rethinking Class-Incremental Learning from a Dynamic Imbalanced Learning Perspective

TL;DR

This work reframes catastrophic forgetting in Class-Incremental Learning as a dynamic imbalance problem and introduces Uniform Prototype Contrastive Learning (UPCL). UPCL learns uniform and compact features by using non-learnable prototypes uniformly distributed on the unit hypersphere $S^{d-1}$, assigns new classes to prototypes with a Hungarian-based alignment, and optimizes a prototype contrastive loss with a dynamic margin $m = -\log p(y)$ that adapts to class priors. The method combines a proto loss and a decaying feature loss to maintain stability of old knowledge while accommodating new classes, achieving state-of-the-art results on CIFAR100, ImageNet100, and TinyImageNet under memory constraints. The work highlights the importance of representation learning under dynamic imbalance for effective lifelong learning, while acknowledging the dimensionality limitation and proposing future work on dynamic prototype dimensionality and unknown class counts.

Abstract

Deep neural networks suffer from catastrophic forgetting when continually learning new concepts. In this paper, we analyze this problem from a data imbalance point of view. We argue that the imbalance between old task and new task data contributes to forgetting of the old tasks. Moreover, the increasing imbalance ratio during incremental learning further aggravates the problem. To address the dynamic imbalance issue, we propose Uniform Prototype Contrastive Learning (UPCL), where uniform and compact features are learned. Specifically, we generate a set of non-learnable uniform prototypes before each task starts. Then we assign these uniform prototypes to each class and guide the feature learning through prototype contrastive learning. We also dynamically adjust the relative margin between old and new classes so that the feature distribution will be maintained balanced and compact. Finally, we demonstrate through extensive experiments that the proposed method achieves state-of-the-art performance on several benchmark datasets including CIFAR100, ImageNet100 and TinyImageNet.

Figures (7)

• Figure 1: The imbalance ratio increase dynamically in CIL and is negatively correlated with memory size. The solid line represents the accuracy on each task, while the dashed line represents the imbalance ratio. The above experiments are performed by iCaRL on CIFAR100.
• Figure 2: Comparison between the imbalanced feature space (a-c) learned by classic incremental learning methods and the uniform feature space (d-f) learned by the proposed UPCL.
• Figure 3: The pipeline of uniform prototype contrastive learning (UPCL). Our method can dynamically generate uniform prototypes on the unit-hypersphere during the CIL process, and heuristically assign classes to the prototypes during the training process, thus ensuring uniformity. Prototype contrastive learning with dynamic margin dynamically adjusts the relative margin between new and old class features in the contrastive loss using the statistical prior of the training data, thus ensuring compactness.
• Figure 4: The performance with different memory sizes. M500, M1000, and M2000 represent the total number of exemplars in memory.
• Figure 5: Confusion matrices in ablation study of each component.