Generalized Category Discovery under Domain Shift: A Frequency Domain Perspective

TL;DR

This work tackles Generalized Category Discovery under domain shift (DS_GCD), where unlabeled data may belong to unseen categories across unknown domains. It introduces FREE, a frequency-domain framework that (i) separates known/unknown domains via FFT amplitude analysis, (ii) bridges domain gaps with cross-domain amplitude swaps and intra-domain perturbations, and (iii) strengthens learning with extended contrastive and clustering losses plus a clustering-difficulty-aware resampling strategy. Through extensive experiments on DomainNet and SSB-C, FREE consistently outperforms state-of-the-art methods, demonstrating robustness to distribution shifts and superior discovery of both known and unknown categories. The approach advances open-world visual recognition by leveraging frequency-domain cues to disentangle semantic structure from domain-specific appearance, with practical implications for robust deployment in non-stationary environments.

Abstract

Generalized Category Discovery (GCD) aims to leverage labeled samples from known categories to cluster unlabeled data that may include both known and unknown categories. While existing methods have achieved impressive results under standard conditions, their performance often deteriorates in the presence of distribution shifts. In this paper, we explore a more realistic task: Domain-Shifted Generalized Category Discovery (DS\_GCD), where the unlabeled data includes not only unknown categories but also samples from unknown domains. To tackle this challenge, we propose a \textbf{\underline{F}}requency-guided Gene\textbf{\underline{r}}alized Cat\textbf{\underline{e}}gory Discov\textbf{\underline{e}}ry framework (FREE) that enhances the model's ability to discover categories under distributional shift by leveraging frequency-domain information. Specifically, we first propose a frequency-based domain separation strategy that partitions samples into known and unknown domains by measuring their amplitude differences. We then propose two types of frequency-domain perturbation strategies: a cross-domain strategy, which adapts to new distributions by exchanging amplitude components across domains, and an intra-domain strategy, which enhances robustness to intra-domain variations within the unknown domain. Furthermore, we extend the self-supervised contrastive objective and semantic clustering loss to better guide the training process. Finally, we introduce a clustering-difficulty-aware resampling technique to adaptively focus on harder-to-cluster categories, further enhancing model performance. Extensive experiments demonstrate that our method effectively mitigates the impact of distributional shifts across various benchmark datasets and achieves superior performance in discovering both known and unknown categories.

Figures (6)

• Figure 1: Illustration of Domain-Shifted GCD setting (DS_GCD) and the traditional GCD setting. In the DS_GCD setting, the model needs to categorize known and unknown categories from both known and unknown domains.
• Figure 2: Clustering accuracy of all categories using FREE, SimGCD+FDA, and SimGCD. It can be observed that FREE outperforms SimGCD+FDA both in final clustering accuracy and convergence speed. In contrast, random transformation in SimGCD+FDA may even hinder the learning of unknown domains, potentially leading to negative transfer effects.
• Figure 3: The overall framework of our proposed FREE method.
• Figure 4: Density histograms for different task combinations. A two-component Gaussian mixture model is fitted to distinguish between known and unknown domain samples (domain labels are used here for visualization only).
• Figure 4: Ablation study on different transformation strategies.