Comparing Euclidean and Hyperbolic K-Means for Generalized Category Discovery

TL;DR

This work tackles open-world Generalized Category Discovery (GCD) by arguing that clustering Euclidean embeddings derived from hyperbolic representations distorts hierarchical structure. It introduces HC-GCD, which learns embeddings in the Lorentz Hyperboloid and clusters directly in hyperbolic space using a novel hyperbolic K-Means, with training guided by contrastive losses in both Euclidean and hyperbolic contexts. The approach achieves parity with state-of-the-art hyperbolic GCD methods on Semantic Shift Benchmark data and generally outperforms Euclidean clustering within the same hyperbolic framework, while ablations show enhanced stability and consistency across label granularity. The findings suggest that native hyperbolic clustering better preserves learned hierarchies, offering practical benefits for open-world classification and potentially more robust hierarchical representations in GCD settings.

Abstract

Hyperbolic representation learning has been widely used to extract implicit hierarchies within data, and recently it has found its way to the open-world classification task of Generalized Category Discovery (GCD). However, prior hyperbolic GCD methods only use hyperbolic geometry for representation learning and transform back to Euclidean geometry when clustering. We hypothesize this is suboptimal. Therefore, we present Hyperbolic Clustered GCD (HC-GCD), which learns embeddings in the Lorentz Hyperboloid model of hyperbolic geometry, and clusters these embeddings directly in hyperbolic space using a hyperbolic K-Means algorithm. We test our model on the Semantic Shift Benchmark datasets, and demonstrate that HC-GCD is on par with the previous state-of-the-art hyperbolic GCD method. Furthermore, we show that using hyperbolic K-Means leads to better accuracy than Euclidean K-Means. We carry out ablation studies showing that clipping the norm of the Euclidean embeddings leads to decreased accuracy in clustering unseen classes, and increased accuracy for seen classes, while the overall accuracy is dataset dependent. We also show that using hyperbolic K-Means leads to more consistent clusters when varying the label granularity.

Figures (4)

• Figure 1: Hyperbolic semi-supervised K-Means in the Lorentz Hyperboloid model of hyperbolic geometry. The centroids are represented as black stars.
• Figure 2: The Lorentz Hyperboloid model $\mathbb{H}^2_1$ as the upper sheet of a hyperboloid. Shown also are geodesics in this model.
• Figure 3: The Klein model $\mathbb{K}^2_1$ as a circle with radius under $1/\kappa$. This model is the projection of the Lorentz Hyperboloid $\mathbb{H}^2_1$ from Figure \ref{['fig:Lorentz-model']}, which shows how the geodesics project into straight lines.
• Figure 4: Full pipeline of HC-GCD. Training uses combination of distance and angle based contrastive loss. After training is finished, a hyperbolic semi-supervised K-Means algorithm is used for clustering the embeddings in the Lorentz Hyperboloid.

Theorems & Definitions (6)

• Lemma 1
• Corollary 1.1
• Theorem 1
• proof
• Lemma 1
• proof