ProtoGuard-guided PROPEL: Class-Aware Prototype Enhancement and Progressive Labeling for Incremental 3D Point Cloud Segmentation

TL;DR

This work tackles catastrophic forgetting in incremental 3D point cloud semantic segmentation by introducing a two-stage framework. ProtoGuard maintains dynamic geometric and semantic prototypes for base classes to enhance discriminative feature learning, while PROPEL freezes the base model and progressively refines pseudo-labels for novel classes using BALD-based uncertainty and density/semantic cues. The approach yields strong gains over baselines on S3DIS and ScanNet, with up to 20.39% mIoU improvement in a 5-step S3DIS scenario and 24.28% on ScanNet, demonstrating effective mitigation of forgetting and improved handling of long-tail and overlapping classes. These results suggest a practical path toward robust, scalable incremental segmentation in dynamic 3D environments.

Abstract

3D point cloud semantic segmentation technology has been widely used. However, in real-world scenarios, the environment is evolving. Thus, offline-trained segmentation models may lead to catastrophic forgetting of previously seen classes. Class-incremental learning (CIL) is designed to address the problem of catastrophic forgetting. While point clouds are common, we observe high similarity and unclear boundaries between different classes. Meanwhile, they are known to be imbalanced in class distribution. These lead to issues including misclassification between similar classes and the long-tail problem, which have not been adequately addressed in previous CIL methods. We thus propose ProtoGuard and PROPEL (Progressive Refinement Of PsEudo-Labels). In the base-class training phase, ProtoGuard maintains geometric and semantic prototypes for each class, which are combined into prototype features using an attention mechanism. In the novel-class training phase, PROPEL inherits the base feature extractor and classifier, guiding pseudo-label propagation and updates based on density distribution and semantic similarity. Extensive experiments show that our approach achieves remarkable results on both the S3DIS and ScanNet datasets, improving the mIoU of 3D point cloud segmentation by a maximum of 20.39% under the 5-step CIL scenario on S3DIS.

Figures (4)

• Figure 1: Visualization of the distribution of each class in ScanNet dai2017scannet and the degree of overlap between classes. We define the degree of overlap between two classes as the ratio of points in one class closer than 0.1 meters to the point cloud of another class.
• Figure 2: Illustration of ProtoGuard. It is divided into two nearly symmetric geometry-semantic streams, each maintaining a geometric prototype and a semantic prototype for each class. These prototypes improve the model's discriminative ability and when combined with edge features, generate more informative features.
• Figure 3: Illustration of PROPEL. The PROPEL framework features a dual-pathway architecture for point cloud CIL. The frozen base model (blue) and trainable novel model (green) process point clouds simultaneously, while our uncertainty estimation module identifies confident (orange) and uncertain (blue) regions. The adaptive threshold dynamically adjusts based on local context, enabling strategic label propagation from confident areas to uncertain regions.
• Figure 4: The visualization of the segmentation results from our method, compared with other CIL methods EWC kirkpatrick2017overcoming, LwF li2017learning, and GFT+UPG yang2023geometry, on S3DIS with $C_{\text{novel}}=5$ and $C_{\text{novel}}=3$ for split = 0, shows that our method's segmentation results are closer to the ground truth.