Boosting the Class-Incremental Learning in 3D Point Clouds via Zero-Collection-Cost Basic Shape Pre-Training

TL;DR

This work tackles catastrophic forgetting in 3D point-cloud class-incremental learning by introducing a zero-collection-cost Basic Shape Dataset (BSA) for geometry-rich pre-training and a geometry-informed CIL-3D framework. The backbone is frozen to preserve learned geometry while adapters learn new object representations, and exemplar-adaptive regularizations build and update class prototypes to resist forgetting in both exemplar-free and exemplar-based settings. Experimental results across ModelNet40, ShapeNet55, and ScanObjectNN show substantial improvements over state-of-the-art baselines, with clear gains in both last-stage and average accuracies and faster convergence when pre-trained on BSA. The approach demonstrates the value of structured geometric pre-training for 3D continual learning, offering a practical and scalable benchmark and method for CIL-3D applications.

Abstract

Existing class-incremental learning methods in 3D point clouds rely on exemplars (samples of former classes) to resist the catastrophic forgetting of models, and exemplar-free settings will greatly degrade the performance. For exemplar-free incremental learning, the pre-trained model methods have achieved state-of-the-art results in 2D domains. However, these methods cannot be migrated to the 3D domains due to the limited pre-training datasets and insufficient focus on fine-grained geometric details. This paper breaks through these limitations, proposing a basic shape dataset with zero collection cost for model pre-training. It helps a model obtain extensive knowledge of 3D geometries. Based on this, we propose a framework embedded with 3D geometry knowledge for incremental learning in point clouds, compatible with exemplar-free (-based) settings. In the incremental stage, the geometry knowledge is extended to represent objects in point clouds. The class prototype is calculated by regularizing the data representation with the same category and is kept adjusting in the learning process. It helps the model remember the shape features of different categories. Experiments show that our method outperforms other baseline methods by a large margin on various benchmark datasets, considering both exemplar-free (-based) settings.

Figures (6)

• Figure 1: In the pre-training stage, the backbone model learns knowledge from the zero-collection-cost basic shapes and shape assemblies. In CIL, the backbone is frozen to remember geometry knowledge, introducing adapters to incremental learn real objects in point clouds.
• Figure 2: The pre-training dataset is generated by shape formulas and specific regulars without any collection cost. A discrete Variational AutoEncoder (dVAE) supervises the predicted tokens of the pre-training dataset. In CIL-task t, the transformer encoder introduces adapter layers to output [CLS] Tokens, cooperating with an exemplar-adaptive regularization method to calculate class prototypes, where A${_t}$-layer-n indicates the adapter layer following the n-th transformer (TF-) layer. In task t+1, TF- and A${_{t-1}}$-layers are frozen, and the learnable A${_t}$-layers update former class prototypes by exemplar-based tuning or exemplar-free mapping method.
• Figure 3: An example of shape assembly. Several basic shapes make a 3D object of our dataset in the point cloud. The object is semantic-agnostic and contains local geometries, which are very similar to those in real objects.
• Figure 4: The classification accuracy ${\mathcal{A}_b}$ at each incremental step with different methods on benchmark datasets (exemplar-based comparisons).
• Figure 5: The classification accuracy ${\mathcal{A}_b}$ at each incremental step with different methods on benchmark datasets (exemplar-free comparisons).