HeAL3D: Heuristical-enhanced Active Learning for 3D Object Detection

TL;DR

HeAL tackles the challenge of label-efficient 3D object detection for autonomous driving by introducing a heuristically informed active-learning strategy. It combines localization- and classification-based uncertainty with augmentation-based inconsistency, using Gaussian mixture models to generate robust heatmaps that guide sample selection. Classwise probability maps and distance/point-density corrections further refine uncertainty estimation, enabling the method to prioritize informative distant or sparse-object samples. Across KITTI and nuScenes, HeAL achieves competitive or superior 3D mAP with substantially fewer labeled examples, highlighting practical impact for scalable real-world deployment and suggesting a complementary multi-strategy AL workflow for diverse data regimes.

Abstract

Active Learning has proved to be a relevant approach to perform sample selection for training models for Autonomous Driving. Particularly, previous works on active learning for 3D object detection have shown that selection of samples in uncontrolled scenarios is challenging. Furthermore, current approaches focus exclusively on the theoretical aspects of the sample selection problem but neglect the practical insights that can be obtained from the extensive literature and application of 3D detection models. In this paper, we introduce HeAL (Heuristical-enhanced Active Learning for 3D Object Detection) which integrates those heuristical features together with Localization and Classification to deliver the most contributing samples to the model's training. In contrast to previous works, our approach integrates heuristical features such as object distance and point-quantity to estimate the uncertainty, which enhance the usefulness of selected samples to train detection models. Our quantitative evaluation on KITTI shows that HeAL presents competitive mAP with respect to the State-of-the-Art, and achieves the same mAP as the full-supervised baseline with only 24% of the samples.

Figures (6)

• Figure 1: Typical driving scenario for an autonomous vehicle. Even for LiDAR detectors, which can measure depth directly, farther away objects are more difficult to localize, estimate and classify because of the distance and the lower number of points they reflect. Such information can be leveraged by Active Learning approaches to find the better samples to be labeled. Even human labelers require the support of camera image to label such examples, which reinforces the idea of difficulty and uncertainty.
• Figure 2: HeAL Score calculation pipeline, presented as BEV for clarity whereas the actual procedure is done on 3D. 1) Our input is a 3D scene represented by a point cloud, which is augmented through a 180 degree rotation around the Z axis perpendicular to the ground. 2) The 3D model from the previous AL iteration is used to detect the bounding boxes on the scene, for both the original and the augmented scene. The augmented detections are rotated again so they can be compared directly with the original detections 3) A GMM is calculated for each one of the classes in the detection, where the variance is determined by the dimensions of the object and the distance to the sensor or the point quantity of the box. 4) The inconsistency between original and augmented GMM is calculated with the KL Divergence. 5) The final HeAL score is the mean KL Divergence across all the classes. For the next AL cycle, the 100 samples with the greatest HeAL score are selected for labeling
• Figure 3: mAP in % with respect to the number of labeled samples for selected AL strategies on KITTI.
• Figure 4: mAP in % with respect to the number of labeled samples for selected AL strategies on Nuscenes.
• Figure 5: mAP in KITTI for each difficulty level