PromptCCD: Learning Gaussian Mixture Prompt Pool for Continual Category Discovery

TL;DR

This work tackles Continual Category Discovery (CCD), where a model must uncover novel categories from a continuously arriving stream of unlabeled data while avoiding catastrophic forgetting. It introduces PromptCCD, a prompt-learning framework that employs a Gaussian Mixture Prompt Pool (GMP) to model data distributions with a Gaussian Mixture Model (GMM) and to generate class-prototype prompts via component means ${_c}$; the GMP also enables prototype replay by sampling from the fitted GMM to preserve past knowledge. The method supports on-the-fly estimation of the number of categories through a split-merge mechanism based on a Hastings ratio, allowing dynamic adaptation when $C$ is unknown. Extensive experiments across generic and fine-grained datasets demonstrate state-of-the-art performance under both known and unknown category counts and introduce continual ACC (cACC) as an appropriate metric for CCD settings. Overall, PromptCCD provides a scalable, prompt-based approach that effectively integrates dynamic category discovery with continual learning using minimal additional parameters.

Abstract

We tackle the problem of Continual Category Discovery (CCD), which aims to automatically discover novel categories in a continuous stream of unlabeled data while mitigating the challenge of catastrophic forgetting -- an open problem that persists even in conventional, fully supervised continual learning. To address this challenge, we propose PromptCCD, a simple yet effective framework that utilizes a Gaussian Mixture Model (GMM) as a prompting method for CCD. At the core of PromptCCD lies the Gaussian Mixture Prompting (GMP) module, which acts as a dynamic pool that updates over time to facilitate representation learning and prevent forgetting during category discovery. Moreover, GMP enables on-the-fly estimation of category numbers, allowing PromptCCD to discover categories in unlabeled data without prior knowledge of the category numbers. We extend the standard evaluation metric for Generalized Category Discovery (GCD) to CCD and benchmark state-of-the-art methods on diverse public datasets. PromptCCD significantly outperforms existing methods, demonstrating its effectiveness. Project page: https://visual-ai.github.io/promptccd .

Figures (7)

• Figure 1: Overview of the Continual Category Discovery task. In the initial stage, the model learns from labelled data, while in the subsequent stages, the model learns from a continuous data stream containing unlabelled instances from known and novel classes.
• Figure 2: Our baseline CCD framework adopts a prompt-based continual learning technique by utilizing a prompt pool module to adapt the vision foundation model for CCD.
• Figure 3: Overview of our proposed PromptCCD framework and Gaussian Mixture Prompting (GMP) module. PromptCCD continually discovers new categories while retaining previously discovered ones by learning a dynamic GMP pool to adapt the vision foundation model for CCD. Specifically, we address CCD by making use of GMP modules to estimate the probability of input $\hat{z_i}$ by calculating the log-likelihood and use the top-k mean of components $\mu_i$ as prompts to guide the foundation model. Lastly, to retain previously learned prompts, we generate prototype samples from the fitted GMM at time step $t - 1$ and fit the current GMM with these samples at time step $t$.
• Figure 4: t-SNE visualization of CIFAR100 with features from our model PromptCCD $w/$GMP and Grow & Merge on each stage.
• Figure 5: Performance curves depicted from Tab. \ref{['tab:ablation_result']} ablation results.