Technical Report for the 5th CLVision Challenge at CVPR: Addressing the Class-Incremental with Repetition using Unlabeled Data -- 4th Place Solution

TL;DR

This work tackles Class-Incremental with Repetition (CIR) in continual visual learning by integrating unlabeled data during training. It combines Learning without Forgetting (LwF) extended to both labeled and unlabeled streams, Less Forgetting Learning (LFL) to preserve feature representations, and a prototype-based pseudo-labeling mechanism that leverages class prototypes to label unlabeled samples. Prototypes are updated across experiences and used with cosine similarity to assign pseudo-labels, enabling the model to retain performance on previously seen classes while accommodating new ones. Experimental results on a 130-class CIR dataset show meaningful improvements over baselines, especially in scenarios where unlabeled data contain prior or future classes, with final accuracy reaching 21.19% and achieving 4th place, highlighting the practical value of unlabeled data in continual learning.

Abstract

This paper outlines our approach to the 5th CLVision challenge at CVPR, which addresses the Class-Incremental with Repetition (CIR) scenario. In contrast to traditional class incremental learning, this novel setting introduces unique challenges and research opportunities, particularly through the integration of unlabeled data into the training process. In the CIR scenario, encountered classes may reappear in later learning experiences, and each experience may involve only a subset of the overall class distribution. Additionally, the unlabeled data provided during training may include instances of unseen classes, or irrelevant classes which should be ignored. Our approach focuses on retaining previously learned knowledge by utilizing knowledge distillation and pseudo-labeling techniques. The key characteristic of our method is the exploitation of unlabeled data during training, in order to maintain optimal performance on instances of previously encountered categories and reduce the detrimental effects of catastrophic forgetting. Our method achieves an average accuracy of 16.68\% during the pre-selection phase and 21.19% during the final evaluation phase, outperforming the baseline accuracy of 9.39%. We provide the implementation code at https://github.com/panagiotamoraiti/continual-learning-challenge-2024 .

Figures (1)

• Figure 1: An overview of our proposed method: Each input image is passed through the model and a frozen instance of the model from the previous experience (old model). In addition to the standard cross-entropy loss between the model's predictions and ground truth labels, two additional losses are introduced to mitigate catastrophic forgetting. The LwF loss penalizes the difference in the logit outputs between the current model and the old model, while the LFL loss encourages feature similarity between the intermediate features of both models. As a final component, a cross-entropy loss is computed using the pseudo-labels generated for the unlabeled data, which are based on saved class prototypes. These representations are updated at the beginning of each experience, where the new ones are computed as the average of the old class prototypes and those from the most recent experience. All losses are combined to form the total loss. The overall method enables the model to effectively leverage both labeled and unlabeled data.