Exemplar-Free Class Incremental Learning via Incremental Representation

TL;DR

This work introduces Incremental Representation (IR), an exemplar-free class incremental learning framework that avoids constructing old pseudo-features by enlarging the feature space through dataset augmentation (rotation or mixup) and stabilizing feature space with a single space-maintenance loss. IR discards the incremental top classifiers and instead uses a nonparametric 1-NN classifier based on class prototypes updated across phases, enabling plastic learning of new classes while mitigating forgetting. Experiments on CIFAR100, TinyImageNet, and ImageNetSubset show that IR achieves competitive incremental accuracy comparable to efCIL methods and significantly better forgetting mitigation, with memory efficiency advantages since no old exemplars are stored. The approach offers a simpler, more interpretable efCIL pathway focused on representation construction rather than complex pseudo-feature generation, making it appealing for privacy-conscious continual learning settings.

Abstract

Exemplar-Free Class Incremental Learning (efCIL) aims to continuously incorporate the knowledge from new classes while retaining previously learned information, without storing any old-class exemplars (i.e., samples). For this purpose, various efCIL methods have been proposed over the past few years, generally with elaborately constructed old pseudo-features, increasing the difficulty of model development and interpretation. In contrast, we propose a \textbf{simple Incremental Representation (IR) framework} for efCIL without constructing old pseudo-features. IR utilizes dataset augmentation to cover a suitable feature space and prevents the model from forgetting by using a single L2 space maintenance loss. We discard the transient classifier trained on each one of the sequence tasks and instead replace it with a 1-near-neighbor classifier for inference, ensuring the representation is incrementally updated during CIL. Extensive experiments demonstrate that our proposed IR achieves comparable performance while significantly preventing the model from forgetting on CIFAR100, TinyImageNet, and ImageNetSubset datasets.

Figures (7)

• Figure 1: Motivation of Incremental Representation, that enlarges the feature space with dataset augmentation strategies, e.g., rotation gidaris2018unsupervised or mixup zhang2018mixup. The enlarged feature space enables the feature extractor to incrementally fit the new tasks without any elaborate old feature construction.
• Figure 2: Framework of Incremental Representation, where AUG and CE are intra-phrase processes while space maintenance is an inter-phases process. AUG enlarges the feature space by augmenting the input dataset with different strategies (e.g., rotation gidaris2018unsupervised or mixup zhang2018mixup), and the transient cross-entropy (CE) plastically enables the intra-phase features discriminative. Space maintenance stably transfers the previous knowledge from the frozen feature extractor to the current trainable one, and negligible storage is used to store the prototypes for the 1-NN classifier.
• Figure 3: Forgetting measures on CIFAR100 for different phrases. Solid lines present efCIL methods, and dashed lines present data replay-based methods.
• Figure 4: Forgetting measures on TinyImageNet for different phrases.
• Figure 5: Forgetting measures on ImageNetSubset for the setting of $10$ incremental phases.