Continual Learning for LiDAR Semantic Segmentation: Class-Incremental and Coarse-to-Fine strategies on Sparse Data

TL;DR

This work tackles continual learning for LiDAR semantic segmentation by introducing class-incremental (CIL) and coarse-to-fine (C2F) partitions on SemanticKITTI and evaluating two architectures, RandLA-Net and Cylinder3D. It adopts Knowledge Distillation (KD) and Background Self-Inpainting to mitigate catastrophic forgetting, with losses that combine $L_{ce}$ and $L_{kd}$ (and optionally $L^{*}_{kd}$) as $\mathcal{L} = L_{ce} + \lambda L_{kd}$, and uses a feature-space distillation $L^{*}_{kd}$ to align representations. The study demonstrates that RandLA-Net is more suited to CL in sparse 3D data, KD (especially at the output level) improves memory of past classes, and self-inpainting provides regularization, though results still lag behind full offline training. Overall, the paper validates the feasibility of class-incremental CL for LiDAR segmentation and lays groundwork for richer partitioning, balancing, and geometry-driven constraints in 3D continual learning.

Abstract

During the last few years, continual learning (CL) strategies for image classification and segmentation have been widely investigated designing innovative solutions to tackle catastrophic forgetting, like knowledge distillation and self-inpainting. However, the application of continual learning paradigms to point clouds is still unexplored and investigation is required, especially using architectures that capture the sparsity and uneven distribution of LiDAR data. The current paper analyzes the problem of class incremental learning applied to point cloud semantic segmentation, comparing approaches and state-of-the-art architectures. To the best of our knowledge, this is the first example of class-incremental continual learning for LiDAR point cloud semantic segmentation. Different CL strategies were adapted to LiDAR point clouds and tested, tackling both classic fine-tuning scenarios and the Coarse-to-Fine learning paradigm. The framework has been evaluated through two different architectures on SemanticKITTI, obtaining results in line with state-of-the-art CL strategies and standard offline learning.

Figures (5)

• Figure 1: Histogram of point clouds percentage within each training dataset split $D_k$. The percentage is normalized for each class $c$ on the total number of points for that specific class in the whole dataset $D_0 \cup D_1 \cup D_2$.
• Figure 2: Class hierarchies in SemanticKITTI C2F partition. Each Coarse class is made-up by grouping classes belonging to the same learning step in CIL partition. Sequence splits $D_k$ are considered in the same identical order.
• Figure 3: Summary of continual learning strategies employed. At the current learning step ($k$) only training set $T_k$ and previous model $\mathcal{M}_{k-1}$ are available.
• Figure 4: Background Self-Inpainting. Labels for old classes $C_{k-1}$ are predicted by previous model $\mathcal{M}_{k-1}$ and Inpainted onto background labels of current training set $T_k$.
• Figure 5: Some qualitative results on CIL with Cylinder3D.