HGL: Hierarchical Geometry Learning for Test-time Adaptation in 3D Point Cloud Segmentation

TL;DR

The paper tackles the challenge of robust test-time adaptation for 3D point cloud segmentation under distribution shift by introducing Hierarchical Geometry Learning (HGL), a bottom-up framework that explicitly exploits local (point-level), global (object-level), and temporal (frame-level) geometric structures. It combines a Local Geometry Learning module for pseudo-label generation, a Global Geometry Fine-tuning module with prototype-based refinement, and a Temporal Geometry Regularization component to enforce cross-frame consistency during online updates. Extensive experiments across synthetic and real datasets, multiple sensor configurations, continual learning, and adverse weather demonstrate state-of-the-art performance and a substantial reduction in adaptation time (up to ~80% faster than prior work). The approach is designed as a general, extensible framework, with substantial improvements over GIPSO and strong robustness to various domain shifts, making it practical for real-world 3D segmentation under changing conditions.

Abstract

3D point cloud segmentation has received significant interest for its growing applications. However, the generalization ability of models suffers in dynamic scenarios due to the distribution shift between test and training data. To promote robustness and adaptability across diverse scenarios, test-time adaptation (TTA) has recently been introduced. Nevertheless, most existing TTA methods are developed for images, and limited approaches applicable to point clouds ignore the inherent hierarchical geometric structures in point cloud streams, i.e., local (point-level), global (object-level), and temporal (frame-level) structures. In this paper, we delve into TTA in 3D point cloud segmentation and propose a novel Hierarchical Geometry Learning (HGL) framework. HGL comprises three complementary modules from local, global to temporal learning in a bottom-up manner.Technically, we first construct a local geometry learning module for pseudo-label generation. Next, we build prototypes from the global geometry perspective for pseudo-label fine-tuning. Furthermore, we introduce a temporal consistency regularization module to mitigate negative transfer. Extensive experiments on four datasets demonstrate the effectiveness and superiority of our HGL. Remarkably, on the SynLiDAR to SemanticKITTI task, HGL achieves an overall mIoU of 46.91\%, improving GIPSO by 3.0\% and significantly reducing the required adaptation time by 80\%. The code is available at https://github.com/tpzou/HGL.

Figures (5)

• Figure 1: Overview of Hierarchical Geometry Learning (HGL) framework. HGL is composed of a local geometry learning (LGL) module, a global geometry fine-tuning (GGF) module, and a temporal geometry regularization module. Specifically, a) LGL exploits local K-NN clustering strategy to generate local geometric pseudo-labels based on neighboring aggregation, prediction certainty, and geometric purity. Then, b) GGF builds the prototype to explore global geometry for pseudo-label fine-tuning. Besides, c) encourages the features of geometrically corresponding points between different frames to remain consistent, where $h( \cdot )$ means encoder network and $f( \cdot )$ means predictor head.
• Figure 2: Hyper-parameter sensitivity analysis on SynLiDAR to SemanticKITTI.
• Figure 3: Visualization of selected pseudo-label of LGL.
• Figure 4: Visualization of selected pseudo-label and adaptation results for SynLiDAR to SemanticKITTI. From left to right: ground-truth label, pseudo-label generated by Source Only model, selected local pseudo-label generated by Our LGL, local-global pseudo-label fine-tuned by Our GGF, and adaptation result predicted by Ours HGL.
• Figure 5: Visual comparison between GIPSO and our HGL. From left to right: target input, ground-truth label, result predicted by source model, adaptation result predicted by GIPSO and our HGL.