An Empirical Study on Knowledge Transfer under Domain and Label Shifts in 3D LiDAR Point Clouds

TL;DR

This work introduces ROAD, a benchmark for evaluating knowledge transfer in 3D LiDAR point clouds under both domain shifts and evolving label taxonomies. It systematically analyzes zero-shot transfer, linear probing, and continual learning across Waymo-to-NuScenes and Waymo-to-Argoverse2 transfers, exploring backbones (PointNet++ vs Point Transformer), base objectives, and CL methods (LwF vs EWC). Key findings show no single architecture dominates; self-supervised objectives help linear probing for certain label shifts but do not consistently improve continual learning, while LwF often outperforms EWC and memory-enhanced schemes can mitigate forgetting. The study provides strong baselines and practical insights for designing robust 3D perception systems, with future directions including stronger backbones, pretraining, and extending to more complex 3D tasks like detection and motion forecasting.

Abstract

For 3D perception systems to be practical in real-world applications -- from autonomous driving to embodied AI -- models must adapt to continuously evolving object definitions and sensor domains. Yet, research on continual and transfer learning in 3D point cloud perception remains underexplored compared to 2D vision -- particularly under simultaneous domain and label shifts. To address this gap, we propose the RObust Autonomous driving under Dataset shifts (ROAD) benchmark, a comprehensive evaluation suite for LiDAR-based object classification that explicitly accounts for domain shifts as well as three key forms of label evolution: class split, class expansion, and class insertion. Using large-scale datasets (Waymo, NuScenes, Argoverse2), we evaluate zero-shot transfer, linear probe, and CL, and analyze the impact of backbone architectures, training objectives, and CL methods. Our findings reveal limitations of existing approaches under realistic shifts and establish strong baselines for future research in robust 3D perception.

Figures (8)

• Figure 1: Overview of our benchmarking framework.
• Figure 2: Continual learning performance on Waymo-to-NuScenes and Waymo-to-Argoverse2 under different $\lambda$ values.
• Figure 3: Continual Learning performance on Waymo-to-NuScenes (Row 1) and Waymo-to-Argoverse2 (Row 2). Column 1 shows LwF models, and Column 2 shows EWC models. Veh., Ped., and Cyc. denote the vehicle, pedestrian, and cyclist classes, respectively.
• Figure 4: Distribution of 3D bounding box dimensions (length, width, height) in Waymo, NuScenes, and Argoverse2 datasets.
• Figure 5: Distribution of important neuron across layers in terms of the importance score in EWC. 1000 most important neurons are selected for each model to draw KDE plots.