Query-aware Hub Prototype Learning for Few-Shot 3D Point Cloud Semantic Segmentation

TL;DR

The paper tackles few-shot 3D point cloud segmentation by addressing prototype bias that arises when prototypes are derived solely from the support set. It introduces Query-aware Hub Prototype (QHP) learning, comprising Hub Prototype Generation (HPG) to create query-relevant prototypes from frequently linked support hubs, and Prototype Distribution Optimization (PDO) to suppress bad hubs via a purity-reweighted contrastive loss. The approach yields more discriminative, query-aligned prototypes and demonstrates state-of-the-art results on S3DIS and ScanNet, validating its effectiveness under distribution shifts. Overall, QHP narrows the semantic gap between prototypes and query data, improving robustness in FS-3DSeg tasks.

Abstract

Few-shot 3D point cloud semantic segmentation (FS-3DSeg) aims to segment novel classes with only a few labeled samples. However, existing metric-based prototype learning methods generate prototypes solely from the support set, without considering their relevance to query data. This often results in prototype bias, where prototypes overfit support-specific characteristics and fail to generalize to the query distribution, especially in the presence of distribution shifts, which leads to degraded segmentation performance. To address this issue, we propose a novel Query-aware Hub Prototype (QHP) learning method that explicitly models semantic correlations between support and query sets. Specifically, we propose a Hub Prototype Generation (HPG) module that constructs a bipartite graph connecting query and support points, identifies frequently linked support hubs, and generates query-relevant prototypes that better capture cross-set semantics. To further mitigate the influence of bad hubs and ambiguous prototypes near class boundaries, we introduce a Prototype Distribution Optimization (PDO) module, which employs a purity-reweighted contrastive loss to refine prototype representations by pulling bad hubs and outlier prototypes closer to their corresponding class centers. Extensive experiments on S3DIS and ScanNet demonstrate that QHP achieves substantial performance gains over state-of-the-art methods, effectively narrowing the semantic gap between prototypes and query sets in FS-3DSeg.

Figures (5)

• Figure 1: Few-shot 3D point cloud semantic segmentation approaches. (a)(b) Previous prototype learning methods generate prototypes solely based on support points. (c) We propose a Query-aware Hub Prototype Learning method that generates prototypes more closely related to query points.
• Figure 2: The framework of our Query-aware Hub Prototype Learning method. Initially, we design an HPG module to select support hubs and generate query-relevant hub prototypes. Moreover, during training, a PDO module is integrated to optimize the distribution of bad hubs and outlier prototypes. For clarity, we present the model under the 1-way 1-shot setting.
• Figure 3: Illustration of Hub Point Mining and Bad Hub Selection modules. We give an example under hyperparameters $k=3$, $\eta=2$, and $\gamma=0.6$. (a)--(c) Hub Point Mining: initially, using center points $\{\mathcal{C}_1, \mathcal{C}_2, \mathcal{C}_3\}$ and neighbor points $\{\mathcal{N}_1, \dots, \mathcal{N}_8\}$ as input, a $k$NN graph is constructed with $k=3$. After calculating hubness scores, the top $\eta=2$ points with the highest hubness scores are selected as hubs. (d) Bad Hub Selection: Hubs with purity below the threshold $\gamma=0.6$ are selected as bad hubs.
• Figure 4: Qualitative comparisons between our proposed model QHP and COSeg. Each row, from top to bottom, represents the 1-way 1-shot task with the target category as chair (yellow), column (blue), bookcase (pink), and ceiling (orange), respectively.
• Figure 5: Parameter sensitivity analysis of HPG and PDO.