The Why, When, and How to Use Active Learning in Large-Data-Driven 3D Object Detection for Safe Autonomous Driving: An Empirical Exploration

TL;DR

This paper addresses the annotation bottleneck for supervised 3D object detection in autonomous driving by applying entropy-based active learning to selectively annotate the most informative unlabeled samples. It evaluates the approach on the nuScenes dataset using the BEVFusion model, comparing entropy querying against random sampling and showing improvements in minority-class performance and overall data efficiency. The study adopts a pool-based, iterative sampling setup with defined budgets and standard 3D detection metrics, demonstrating that entropy querying can reduce labeling costs while maintaining or enhancing accuracy. Limitations include a single experimental run, and the authors advocate future work on learning data-query policies via reinforcement learning to further optimize active learning in safety-critical driving scenarios.

Abstract

Active learning strategies for 3D object detection in autonomous driving datasets may help to address challenges of data imbalance, redundancy, and high-dimensional data. We demonstrate the effectiveness of entropy querying to select informative samples, aiming to reduce annotation costs and improve model performance. We experiment using the BEVFusion model for 3D object detection on the nuScenes dataset, comparing active learning to random sampling and demonstrating that entropy querying outperforms in most cases. The method is particularly effective in reducing the performance gap between majority and minority classes. Class-specific analysis reveals efficient allocation of annotated resources for limited data budgets, emphasizing the importance of selecting diverse and informative data for model training. Our findings suggest that entropy querying is a promising strategy for selecting data that enhances model learning in resource-constrained environments.

Figures (7)

• Figure 1: Novel safety-critical events occur with low probability while driving, making data collection of such events an enormous cost, especially since the number of instances required to teach a high-dimensional model scales exponentially with the number of data dimensions. While the left region of this curve may represent scenarios encountered in normal driving, as we progress to the right, we would expect to find not only unexpected driving environments and interactions, but also those of near-miss accidents and catastrophic failures. Collecting real-world data on dangerous accidents (and, at that, sufficient instances of this data to build models via supervised learning in high-dimensional vector spaces) is an extremely challenging task. The blue curve carries the moniker of "long-tail events".
• Figure 2: The amount of carefully annotated data available during training is closely tied to the success of the learned model. This is an image from the nuScenes dataset, whose camera and LiDAR measurements are used as input to the BEVFusion 3D Detection model discussed in this paper. When the model is trained with 10% of the available training data, we can see a high rate of false positive detections throughout the scene, and failure to note even the obvious-but-partially-occluded vehicle. As we increase the training data to 35% of the available pool, under random sampling, the false positive detections remain confounding, but the pedestrians on the sidewalk are missed altogether, and there is a general difficulty to capture the precise position, size, and orientation of these objects. On the other hand, when using the entropy querying active learning method detailed in this paper, under the same data budget, the pedestrian on the sidewalk is found and the false positive detections are significantly reduced relative to the ground truth. The ground truth, depicted on right, shows the ideal detection, which requires the careful selection of additional data points to further boost trained model performance without incurring expensive demand for extensive data annotation. In this research, we present methods for intelligently querying the available data pool for new training samples using active learning.
• Figure 3: An illustration of Active Learning setup with the BEVFusion model.
• Figure 4: Overview of per-class results, with Random Sampling on left and Entropy Querying on right. While the ordering of classes remains intact and nearly identical to the frequency of appearance of respective classes in the dataset, under entropy sampling, the margin between best and worst performing classes decreases.
• Figure 5: Class-separated analysis of mAP performance between random sampling and entropy query active learning. Entropy query active learning shows a tendency to outperform random sampling on mAP, shown on the six minority classes in these graphs.