Exemplar-free Class Incremental Learning via Discriminative and Comparable One-class Classifiers

TL;DR

Exemplar-free Class-IL requires learning new classes without retaining old samples, creating discrimination and cross-class comparability challenges. The authors propose DisCOIL, a parallel-VAE framework where each class has a dedicated VAE that acts as a one-class classifier and a generator of old-class pseudo-samples; discriminability is enforced with a hinge reconstruction loss, and comparability is achieved via classifier-contrastive and inter-class losses, leveraging generated negatives for learning. The approach expands with new tasks and demonstrates state-of-the-art performance on MNIST, CIFAR-10, and Tiny-ImageNet, with ablations confirming the contributions of each loss term and the pseudo-sample mechanism. The work highlights the potential of combining generation with one-class learning to mitigate forgetting in privacy-preserving incremental scenarios, while noting parameter growth and suggesting integration with parameter-sharing and distribution-alignment techniques for scalability.

Abstract

The exemplar-free class incremental learning requires classification models to learn new class knowledge incrementally without retaining any old samples. Recently, the framework based on parallel one-class classifiers (POC), which trains a one-class classifier (OCC) independently for each category, has attracted extensive attention, since it can naturally avoid catastrophic forgetting. POC, however, suffers from weak discriminability and comparability due to its independent training strategy for different OOCs. To meet this challenge, we propose a new framework, named Discriminative and Comparable One-class classifiers for Incremental Learning (DisCOIL). DisCOIL follows the basic principle of POC, but it adopts variational auto-encoders (VAE) instead of other well-established one-class classifiers (e.g. deep SVDD), because a trained VAE can not only identify the probability of an input sample belonging to a class but also generate pseudo samples of the class to assist in learning new tasks. With this advantage, DisCOIL trains a new-class VAE in contrast with the old-class VAEs, which forces the new-class VAE to reconstruct better for new-class samples but worse for the old-class pseudo samples, thus enhancing the comparability. Furthermore, DisCOIL introduces a hinge reconstruction loss to ensure the discriminability. We evaluate our method extensively on MNIST, CIFAR10, and Tiny-ImageNet. The experimental results show that DisCOIL achieves state-of-the-art performance.

Figures (6)

• Figure 1: Ideal incremental learning results with two one-class classifiers (OCC). Following the vertical arrow viewpoint, both OCCs should have discriminability to determine whether an input sample belongs to their corresponding class. Following the horizontal arrow viewpoint, they need to have comparability that allows the outputs of the two OCCs to be compared to each other.
• Figure 2: An example of misclassification that caused by two non-comparable one-class classifiers. The two curves represent the output distribution of the two discriminative one-class classifiers.
• Figure 3: The architecture of DisCOIL based on variational auto-encoder (VAE).
• Figure 4: Diagram of the training process in DisCOIL. Assuming we are training the VAE corresponding to class "3", and the VAEs of class "1" and class "2" are trained before. For positive samples, we force the reconstruction errors to be lower than a pre-defined threshold $r_{intra}$; for negative samples, we require the reconstruction error to be greater than a pre-defined threshold $r_{inter}$. Besides, we also send the positive samples into old-task VAEs and force the new-class VAE to output lower anomaly scores than the old-task VAEs.
• Figure 5: Sensitivity analysis of $r_{intra}$ (a) and $r_{inter}$ (b) on CIFAR10.