Generalized Fine-Grained Category Discovery with Multi-Granularity Conceptual Experts

TL;DR

This paper tackles Generalized Category Discovery (GCD) in open-world settings by introducing MGCE, a framework that jointly discovers categories and learns discriminative representations without requiring the number of novel classes in advance. MGCE comprises Dynamic Conceptual Contrastive Learning (DCCL), which iteratively generates concepts and learns instance- and concept-level representations, and Multi-Granularity Experts Collaborative Learning (MECL), which uses multiple concept experts at different granularities and ensures cross-granularity alignment. The approach automatically estimates the unlabeled class count $K_U$ and leverages a Semi-Infomap-based concept generation with adaptive neighborhood sizes, plus a concept memory buffer for prototypes. Experiments on nine fine-grained benchmarks (and generic datasets) show state-of-the-art results, with MGCE particularly excelling in novel-class accuracy and robustness to unknown $K_U$, demonstrating practical impact for open-world recognition and scalable clustering.

Abstract

Generalized Category Discovery (GCD) is an open-world problem that clusters unlabeled data by leveraging knowledge from partially labeled categories. A key challenge is that unlabeled data may contain both known and novel categories. Existing approaches suffer from two main limitations. First, they fail to exploit multi-granularity conceptual information in visual data, which limits representation quality. Second, most assume that the number of unlabeled categories is known during training, which is impractical in real-world scenarios. To address these issues, we propose a Multi-Granularity Conceptual Experts (MGCE) framework that adaptively mines visual concepts and integrates multi-granularity knowledge for accurate category discovery. MGCE consists of two modules: (1) Dynamic Conceptual Contrastive Learning (DCCL), which alternates between concept mining and dual-level representation learning to jointly optimize feature learning and category discovery; and (2) Multi-Granularity Experts Collaborative Learning (MECL), which extends the single-expert paradigm by introducing additional experts at different granularities and by employing a concept alignment matrix for effective cross-expert collaboration. Importantly, MGCE can automatically estimate the number of categories in unlabeled data, making it suitable for practical open-world settings. Extensive experiments on nine fine-grained visual recognition benchmarks demonstrate that MGCE achieves state-of-the-art results, particularly in novel-class accuracy. Notably, even without prior knowledge of category numbers, MGCE outperforms parametric approaches that require knowing the exact number of categories, with an average improvement of 3.6\%. Code is available at https://github.com/HaiyangZheng/MGCE.

Figures (8)

• Figure 1: Pipeline of Generalized Category Discovery (GCD). GCD aims to exploit knowledge from labeled data to automatically cluster unlabeled data, which contain both known and unknown categories.
• Figure 2: (a) Schematic illustration of the proposed dynamic category discovery process. (b) Schematic illustration of perspectives at different information granularities. (c) Framework of the proposed Dynamic Conceptual Contrastive Learning (DCCL) module, which incorporates Dynamic Concept Generation and Dual-Level Representation Learning. (d) Framework of the proposed Multi-Granularity Experts Collaborative Learning (MECL) module, which introduces multiple views at different levels of information granularity.
• Figure 3: The MGCE framework consists of two principal modules: Dynamic Conceptual Contrastive Learning (DCCL) and Multi-Granularity Experts Collaborative Learning (MECL). DCCL alternates between Dynamic Concept Generation (DCG) and Dual-level Representation Learning DRL) to form a conceptual expert. Building on DCCL, MECL integrates two components: Multi-Expert Conceptual Learning (ME), which constructs expert branches with varying levels of information granularity, and Collaborative Learning (CL), which aligns and integrates their insights. This enables experts to be optimized collectively and to mutually benefit from each another.
• Figure 4: Comparison of All ACC between multi-expert and single-expert learning across five datasets.
• Figure 5: Evolution of All ACC and the number of effective categories during training on CUB, for single-expert and multi-expert learning.