Breaking the SSL-AL Barrier: A Synergistic Semi-Supervised Active Learning Framework for 3D Object Detection

TL;DR

This work tackles the annotation bottleneck in LiDAR-based 3D object detection by integrating semi-supervised learning with active learning through a synergistic framework (S-SSAL). It introduces CPSP to pre-train on pseudo-scenes formed from confident objects, avoiding noise from uncertain labels, and CAL to optimize unlabeled-data selection via ensemble uncertainty, diversity, and class-balanced sampling. Empirical results on KITTI and Waymo show state-of-the-art performance with minimal labeling—e.g., KITTI with only 2% labeled data matching full-data models—and demonstrate robustness across rounds and rare classes. The approach offers a practical path to scalable 3D detection in autonomous driving by more effectively leveraging unlabeled data while maintaining reliable uncertainty estimates and balanced class coverage.

Abstract

To address the annotation burden in LiDAR-based 3D object detection, active learning (AL) methods offer a promising solution. However, traditional active learning approaches solely rely on a small amount of labeled data to train an initial model for data selection, overlooking the potential of leveraging the abundance of unlabeled data. Recently, attempts to integrate semi-supervised learning (SSL) into AL with the goal of leveraging unlabeled data have faced challenges in effectively resolving the conflict between the two paradigms, resulting in less satisfactory performance. To tackle this conflict, we propose a Synergistic Semi-Supervised Active Learning framework, dubbed as S-SSAL. Specifically, from the perspective of SSL, we propose a Collaborative PseudoScene Pre-training (CPSP) method that effectively learns from unlabeled data without introducing adverse effects. From the perspective of AL, we design a Collaborative Active Learning (CAL) method, which complements the uncertainty and diversity methods by model cascading. This allows us to fully exploit the potential of the CPSP pre-trained model. Extensive experiments conducted on KITTI and Waymo demonstrate the effectiveness of our S-SSAL framework. Notably, on the KITTI dataset, utilizing only 2% labeled data, S-SSAL can achieve performance comparable to models trained on the full dataset. The code has been released at https://github.com/LandDreamer/S_SSAL.

Figures (9)

• Figure 1: The illustration depicts different paradigms for combining Active Learning (AL) and Supervised/Semi-supervised Learning (SL/SSL): (a) Solely utilizing SL in all stages. (b) Employing SSL only in the final model delivering stage. (c) Integrating SSL across all stages. Paradigm (c) achieves enhanced performance by incorporating unlabeled data compared to paradigm (a). However, traditional SSL methods face conflicts with AL in the temporary model updating stage, leading to suboptimal data selection. Thus, paradigm (c) performs less effectively than paradigm (b).
• Figure 2: Overview of our S-SSAL framework. In the Temporary Model Updating stage(TMU), we propose creating pseudo scenes with confident objects for model pre-training (CPSP). Subsequently, in the Unlabeled Sample Selecting stage(USS), we design a collaborative active learning method to select valuable data for annotation (CAL). Finally, in the Final Model Delivering stage(FMD), we leverage traditional semi-supervised learning methods to enhance the model performance.
• Figure 3: The illustration of the Collaborative PseudoScene Pre-training (CPSP) module. We extract confident objects from unlabeled scenes based on their uncertainty and store them in a box bank, which is iteratively updated to maintain its quality. Additionally, we remove point clouds corresponding to the predicted boxes, creating "background" scenes without any objects. The point cloud from the box bank is then inserted into these "background" scenes, forming pseudo scenes.
• Figure 4: This figure showcases the visualization of selected samples without dropping the "DontCare" cases. It displays the ground-truth (GT) boxes in green, the predicted boxes in red, and the "DontCare" areas in white. Each scene is presented through a 2D image and a point cloud representation. In the 2D images, both the GT boxes and the "DontCare" areas are visualized, while in the point cloud scenes, both the GT and predicted boxes are visualized.
• Figure 5: Visualization of different pre-training methods. Green represents GT boxes, while red indicates predicted boxes, with numbers denoting confidence scores. In Normal, an unconfident car and a false positive pedestrian (tree) are detected. UNC is impacted by noise from unconfident labels, leading to more false positives. In contrast, CON learns the car effectively and eliminates the false positives.