ReLKD: Inter-Class Relation Learning with Knowledge Distillation for Generalized Category Discovery

TL;DR

<3-5 sentence high-level summary> Generalized Category Discovery (GCD) requires labeling unknown classes present in unlabeled data while known classes have labels, but existing approaches overlook inter-class relations. ReLKD introduces an end-to-end framework with three modules: a Target-Grained Module for discriminative target-level representations, a Coarse-Grained Module to learn implicit hierarchical relations via pseudo-super-class labels, and a Knowledge Distillation Module that transfers cross-level relational knowledge from coarse to target levels. By leveraging implicit inter-class relations and hierarchical structure, ReLKD achieves state-of-the-art performance on four diverse datasets and shows robust novel-class discovery with limited labeled data. The work demonstrates the practical value of incorporating hierarchical information and cross-level distillation in GCD, and provides code for reproducibility.

Abstract

Generalized Category Discovery (GCD) faces the challenge of categorizing unlabeled data containing both known and novel classes, given only labels for known classes. Previous studies often treat each class independently, neglecting the inherent inter-class relations. Obtaining such inter-class relations directly presents a significant challenge in real-world scenarios. To address this issue, we propose ReLKD, an end-to-end framework that effectively exploits implicit inter-class relations and leverages this knowledge to enhance the classification of novel classes. ReLKD comprises three key modules: a target-grained module for learning discriminative representations, a coarse-grained module for capturing hierarchical class relations, and a distillation module for transferring knowledge from the coarse-grained module to refine the target-grained module's representation learning. Extensive experiments on four datasets demonstrate the effectiveness of ReLKD, particularly in scenarios with limited labeled data. The code for ReLKD is available at https://github.com/ZhouF-ECNU/ReLKD.

Figures (4)

• Figure 1: An illustrative example of implicit hierarchical inter-class relations across categories.
• Figure 2: The architecture of ReLKD.
• Figure 3: Generation of a pseudo super-class label for a labeled instance.
• Figure 4: (a) Coarse-grained ACC. (b-d) Impact of novel ratio on All, Seen, and Novel classes. (e) Impact of loss weights on Novel ACC.