TACLE: Task and Class-aware Exemplar-free Semi-supervised Class Incremental Learning

TL;DR

This work tackles exemplar-free semi-supervised class incremental learning (EFSS-CIL) by introducing TACLE, a two-stage framework that leverages pre-trained backbones to learn from labeled and unlabeled data without storing past exemplars. It introduces three core components: a task-wise adaptive threshold for unlabeled data, a class-aware weighted cross-entropy loss to mitigate within-task class imbalance, and classifier alignment that exploits confident unlabeled data via Gaussian class statistics across all seen tasks. Empirical results on CIFAR10, CIFAR100, and ImageNet-Subset100 demonstrate substantial improvements over competitive baselines, especially under low supervision and in one-shot or imbalanced scenarios. The approach enables strong incremental performance while preserving privacy and storage, with practical implications for continual learning in settings where exemplars cannot be retained.

Abstract

We propose a novel TACLE (TAsk and CLass-awarE) framework to address the relatively unexplored and challenging problem of exemplar-free semi-supervised class incremental learning. In this scenario, at each new task, the model has to learn new classes from both (few) labeled and unlabeled data without access to exemplars from previous classes. In addition to leveraging the capabilities of pre-trained models, TACLE proposes a novel task-adaptive threshold, thereby maximizing the utilization of the available unlabeled data as incremental learning progresses. Additionally, to enhance the performance of the under-represented classes within each task, we propose a class-aware weighted cross-entropy loss. We also exploit the unlabeled data for classifier alignment, which further enhances the model performance. Extensive experiments on benchmark datasets, namely CIFAR10, CIFAR100, and ImageNet-Subset100 demonstrate the effectiveness of the proposed TACLE framework. We further showcase its effectiveness when the unlabeled data is imbalanced and also for the extreme case of one labeled example per class.

Figures (11)

• Figure 1: Difference between Class Incremental Learning (CIL), Semi-Supervised CIL (SS-CIL), and Exemplar-Free Semi-Supervised CIL (EFSS-CIL) settings.
• Figure 2: Illustrates the Average Confidence Score (ACS) for unlabeled data across tasks. The ACS calculated by taking average of maximum probability confidence scores from all the unlabeled data, at the end of training. The observed decaying trend indicates that using a fixed high threshold in SS-CIL may not be suitable for effective utilization of unlabeled data in feature learning. Due to the fixed threshold, the amount of unlabeled data utilized for training is significantly reduced as tasks progresses.
• Figure 3: The proposed TACLE introduce two components in stage 1 training at task $t$: C1. Task-wise adaptive threshold ($\gamma_{a}^{(t)}$) is employed in the computation of the unsupervised loss $\mathcal{L}_{us}$. C2. Class-aware weights are utilized in the computation of both supervised and unsupervised losses, where the weights are determined based on the class-wise distribution of pseudo-unlabeled data.
• Figure 4: After stage 1 training, we filter out under-confident samples and create the expanded label set $\tilde{\mathcal{D}}^{(t)} = \mathcal{D}_{l}^{(t)} \cup \tilde{\mathcal{D}}_{ul}^{(t)}$. We estimate class statistics for task $t$ using this expanded label set. Utilizing class-wise statistics for all encountered classes, we fine-tune all classifiers with the classifier alignment loss $\mathcal{L}_{ca}$, defined in Eq. \ref{['eq_l_ca']}. This comprehensive strategy which effectively utilizes the unlabeled data, constitutes our third component (C3) in the proposed approach.
• Figure 5: Analysis of one-shot SS-CIL and imbalance SS-CIL experiments. Experiments were conducted on CIFAR100 ($0.8\%$ labeled data for imbalance scenerio) with 10 tasks, reporting top-1 cumulative accuracy at the end of each task and average cumulative accuracy at the end of each plot. Results are presented for both pre-trained models.