Beyond Known Clusters: Probe New Prototypes for Efficient Generalized Class Discovery

TL;DR

This work tackles Generalized Category Discovery by addressing two core challenges: insufficient prototypes due to underestimation of cluster counts and the computational burden of clustering all data. It introduces Probing New Prototype (PNP), an adaptive, end-to-end framework that expands cluster prototypes with learnable potential prototypes and uses a self-distilled learning strategy to optimize them without ground-truth labels. By clustering only unlabelled data and leveraging labelled data through prototypical learning, PNP achieves state-of-the-art results among one-stage GCD methods and delivers substantial efficiency gains, notably 12× faster clustering on Herbarium 19 and large accuracy improvements on Stanford Cars. The approach is validated across diverse datasets and exposed through extensive ablations and analyses, demonstrating robust improvements in representation learning, prototype expansion, and open-world class discovery.

Abstract

Generalized Class Discovery (GCD) aims to dynamically assign labels to unlabelled data partially based on knowledge learned from labelled data, where the unlabelled data may come from known or novel classes. The prevailing approach generally involves clustering across all data and learning conceptions by prototypical contrastive learning. However, existing methods largely hinge on the performance of clustering algorithms and are thus subject to their inherent limitations. Firstly, the estimated cluster number is often smaller than the ground truth, making the existing methods suffer from the lack of prototypes for comprehensive conception learning. To address this issue, we propose an adaptive probing mechanism that introduces learnable potential prototypes to expand cluster prototypes (centers). As there is no ground truth for the potential prototype, we develop a self-supervised prototype learning framework to optimize the potential prototype in an end-to-end fashion. Secondly, clustering is computationally intensive, and the conventional strategy of clustering both labelled and unlabelled instances exacerbates this issue. To counteract this inefficiency, we opt to cluster only the unlabelled instances and subsequently expand the cluster prototypes with our introduced potential prototypes to fast explore novel classes. Despite the simplicity of our proposed method, extensive empirical analysis on a wide range of datasets confirms that our method consistently delivers state-of-the-art results. Specifically, our method surpasses the nearest competitor by a significant margin of 9.7% within the Stanford Cars dataset and 12x clustering efficiency within the Herbarium 19 dataset. We will make the code and checkpoints publicly available at https://github.com/xjtuYW/PNP.git.

Figures (7)

• Figure 1: This illustration delineates the distinctions among various methods, where each circle represents an instance, diverse colors symbolize different classes or groups, and the pentagram means the cluster prototype (center). In contrast to (a) existing one-stage methods pu2023dynamiczhao2023learning, a salient distinction of our proposed method is (b) the incorporation of potential prototypes (Pot. Proto.) to effectively identify a broader spectrum of novel classes.
• Figure 2: Illustration of PNP framework. The PNP framework adeptly acquires conception-level representations via potential prototype probing and further refines these representations through self-distillation learning. Initially, the potential prototype probing learns conception-level representation with $\mathcal{D}^{u}$ by prototype expansion and self-distilled prototype representation learning. Then, it learns conception-level representation with $\mathcal{D}^{l}$ by setting trainable prototypes and prototypical contrastive learning. After the training process, the framework retains solely the encoder of the student prober.
• Figure 3: (left:) The overall clustering accuracy given by different sizes of memory buffer, where $|\mathcal{Y}^l|$ indicates the category number of labelled data. (right:) The overall clustering accuracy (ALL) given by different numbers of neighbors used to filter the graph in the clustering process.
• Figure 4: t-SNE visualization of instances in CUB200 for features generated by (a) removing and (b) using potential prototypes (PP), where $K^e$ refers to the estimated class or cluster number.
• Figure 5: Prediction biases between Old/New classes with/without using potential prototypes (PP).