Large-scale Pre-trained Models are Surprisingly Strong in Incremental Novel Class Discovery

TL;DR

This paper tackles class-incremental Novel Class Discovery (class-iNCD) without any labelled data, proposing a paradigm that leverages self-supervised pre-trained models (PTMs) to provide strong priors for continual clustering. It introduces two simple baselines, Baseline and Baseline++, built on a frozen PTM backbone (e.g., DINO with ViT-B/16) and a cosine-normalized linear classifier, with Sinkhorn-Knopp pseudo-labeling guiding self-supervised discovery. Baseline++ further employs Knowledge Transfer with Robust Feature Replay (KTRFR) to replay past feature prototypes and maintain discrimination across tasks, improving performance as the number of steps grows. Across five datasets and multi-step sequences, the baselines outperform adapted state-of-the-art methods, often approaching joint-training upper bounds, and highlight the practical value of strong, simple baselines in class-iNCD. The work includes open-source code to foster reproducibility and future research in unsupervised, continual discovery without reliance on labelled data.

Abstract

Discovering novel concepts in unlabelled datasets and in a continuous manner is an important desideratum of lifelong learners. In the literature such problems have been partially addressed under very restricted settings, where novel classes are learned by jointly accessing a related labelled set (e.g., NCD) or by leveraging only a supervisedly pre-trained model (e.g., class-iNCD). In this work we challenge the status quo in class-iNCD and propose a learning paradigm where class discovery occurs continuously and truly unsupervisedly, without needing any related labelled set. In detail, we propose to exploit the richer priors from strong self-supervised pre-trained models (PTM). To this end, we propose simple baselines, composed of a frozen PTM backbone and a learnable linear classifier, that are not only simple to implement but also resilient under longer learning scenarios. We conduct extensive empirical evaluation on a multitude of benchmarks and show the effectiveness of our proposed baselines when compared with sophisticated state-of-the-art methods. The code is open source.

Figures (11)

• Figure 1: Overview of different learning paradigms for discovering novel (or new) categories from unlabelled data. (a) NCD learns and discovers novel classes in an unalabelled dataset by exploiting the priors learned from related labelled data. (b) class-iNCD is similar to NCD, except it discovers novel classes arriving in sessions without any access to labelled data during the discovery phase. (c) Our proposed simple Baseline for class-iNCD that leverages a self-supervised pre-trained model (PTM) instead of expensive labelled data. Inference on test data is carried out in a task-agnostic manner.
• Figure 2: Comparison of traditional Supervised pre-training (Sup.) with self-supervised pre-trained model (PTM) initialization on the Novel Class Discovery.
• Figure 3: Comparison of our proposed baselines with the incremental learning (EwC, LwF, DER), unsupervised incremental learning (CaSSLe), and iNCD (ResTune, FRoST) methods on CIFAR-100. In each step 20 novel classes are learned. We report the Overall Accuracy and Maximum Forgetting.
• Figure 4: Overview framework of the proposed methods Baseline and Baseline++ for class-iNCD task.
• Figure 5: Generalizability analysis. Results are reported on the five-step split of C100 with DINO-ViT-B/16.