Exploring Diversity-based Active Learning for 3D Object Detection in Autonomous Driving

TL;DR

This work investigates diversity-based active learning (AL) as a potential solution to alleviate the annotation burden, and proposes a novel acquisition function that enforces spatial and temporal diversity in the selected samples.

Abstract

3D object detection has recently received much attention due to its great potential in autonomous vehicle (AV). The success of deep learning based object detectors relies on the availability of large-scale annotated datasets, which is time-consuming and expensive to compile, especially for 3D bounding box annotation. In this work, we investigate diversity-based active learning (AL) as a potential solution to alleviate the annotation burden. Given limited annotation budget, only the most informative frames and objects are automatically selected for human to annotate. Technically, we take the advantage of the multimodal information provided in an AV dataset, and propose a novel acquisition function that enforces spatial and temporal diversity in the selected samples. We benchmark the proposed method against other AL strategies under realistic annotation cost measurement, where the realistic costs for annotating a frame and a 3D bounding box are both taken into consideration. We demonstrate the effectiveness of the proposed method on the nuScenes dataset and show that it outperforms existing AL strategies significantly. Code is available at https://github.com/Linkon87/Exploring-Diversity-based-Active-Learning-for-3D-Object-Detection-in-Autonomous-Driving

Figures (8)

• Figure 1: Distribution of samples selected by different active learning strategies and the corresponding performance - (a) Entropy, (b) Feature Diversity, (c) Temporal Diversity, (d) Spatial Diversity and (e) Combined Spatial and Temporal Diversity. (f) Performance curves on nuScenes. Colors represent different selection strategies. The black-white color scale along the path represents original data distribution density (black indicates high density). The proposed spatial and temporal diversity is able to select a diverse of samples without the need of a good feature extractor, while uncertainty-based sampling tends to concentrate in regions of high data density, resulting in redundant samples being selected.
• Figure 2: We illustrate the pipeline of active learning for 3D object detection on 9 sample frames $T_0$ to $T_8$. For all available data, (a) Spatial and (b) Temporal information are directly extracted from metadata. (c) Feature information is computed as the global average pooling of the feature map produced by the backbone network. We first initialize the active learning cycle with limited labeled data and large unlabeled data. Unlabeled samples are then chosen by Eq. \ref{['EqKCenter']} for annotation. The annotated sample is appended to the labeled pool for model training. The cycle is repeated until labeling budget is exhausted.
• Figure 3: An illustration of the k-Center objective. Two instances (red outlined circles) are labeled and another two instances (green outlined circles) are the candidates to be selected. The distance indicated by red line is the maximal minimal distance to be optimized by Eq. \ref{['EqKCenter']}. Choosing which two candidate instances (green outlined circles) are the optimization variables.
• Figure 4: Distribution of selected bounding boxes over categories at budget 4800.
• Figure 5: Number of frames selected by different strategies at fixed budgets.