Active Generalized Category Discovery

TL;DR

An adaptive sampling strategy is devised, which jointly considers novelty, informativeness and diversity to adaptively select novel samples with proper uncertainty to improve the performance of GCD by actively selecting a limited amount of valuable samples for labeling from the oracle.

Abstract

Generalized Category Discovery (GCD) is a pragmatic and challenging open-world task, which endeavors to cluster unlabeled samples from both novel and old classes, leveraging some labeled data of old classes. Given that knowledge learned from old classes is not fully transferable to new classes, and that novel categories are fully unlabeled, GCD inherently faces intractable problems, including imbalanced classification performance and inconsistent confidence between old and new classes, especially in the low-labeling regime. Hence, some annotations of new classes are deemed necessary. However, labeling new classes is extremely costly. To address this issue, we take the spirit of active learning and propose a new setting called Active Generalized Category Discovery (AGCD). The goal is to improve the performance of GCD by actively selecting a limited amount of valuable samples for labeling from the oracle. To solve this problem, we devise an adaptive sampling strategy, which jointly considers novelty, informativeness and diversity to adaptively select novel samples with proper uncertainty. However, owing to the varied orderings of label indices caused by the clustering of novel classes, the queried labels are not directly applicable to subsequent training. To overcome this issue, we further propose a stable label mapping algorithm that transforms ground truth labels to the label space of the classifier, thereby ensuring consistent training across different active selection stages. Our method achieves state-of-the-art performance on both generic and fine-grained datasets. Our code is available at https://github.com/mashijie1028/ActiveGCD

Figures (12)

• Figure 1: The diagram of three settings. Left: Conventional AL is a closed-world setting, where labeled and unlabeled classes are identical. Middle: GCD requires no active labeling and suffers from severe issues. Right: AGCD is an open-world extrapolated version of AL, where unlabeled data contains novel categories, and models are trained on both labeled and unlabeled data to cluster both old and new classes.
• Figure 2: Accuracy of old and new classes in AGCD with different methods (shapes) on various datasets (colors). The closer to the diagonal, the more balanced accuracy between old and new classes. In each dataset (color), our method (star) achieves not only the best overall performance but also more balanced accuracy.
• Figure 3: Accuracy of old and new classes on six datasets.
• Figure 4: Confidence distribution on ImageNet-100. Three measures: -entropy (a), margin (b) and MSP (c).
• Figure 5: The framework of AGCD. (a) Overall pipeline and dataflow. Models are trained on $\mathcal{D}_l^t\cup\mathcal{D}_u^t$ with SimGCD, and select samples in $\mathcal{D}_u^t$. (b) The proposed Adaptive-Novel sampling strategy. Here $\mathcal{M}$ denotes the label mapping function. Stable $\mathcal{M}$ means that at the initial and end epochs of the current round, $\mathcal{M}$ does not change largely. Confident novel samples are sampled at early rounds, when $\mathcal{M}$ is stable, we select the informative ones. (c) Illustration of label mapping computed by model predictions and ground truth on $\mathcal{D}_l^{t-1}\cup\mathcal{D}_q^t$.