MEDL-U: Uncertainty-aware 3D Automatic Annotation based on Evidential Deep Learning

TL;DR

This paper proposes MEDL-U, an EDL framework based on MTrans, which not only generates pseudo labels but also quantifies the associated uncertainties, and achieves state-of-the-art results on the KITTI official test set compared to existing 3D automatic annotators.

Abstract

Advancements in deep learning-based 3D object detection necessitate the availability of large-scale datasets. However, this requirement introduces the challenge of manual annotation, which is often both burdensome and time-consuming. To tackle this issue, the literature has seen the emergence of several weakly supervised frameworks for 3D object detection which can automatically generate pseudo labels for unlabeled data. Nevertheless, these generated pseudo labels contain noise and are not as accurate as those labeled by humans. In this paper, we present the first approach that addresses the inherent ambiguities present in pseudo labels by introducing an Evidential Deep Learning (EDL) based uncertainty estimation framework. Specifically, we propose MEDL-U, an EDL framework based on MTrans, which not only generates pseudo labels but also quantifies the associated uncertainties. However, applying EDL to 3D object detection presents three primary challenges: (1) relatively lower pseudolabel quality in comparison to other autolabelers; (2) excessively high evidential uncertainty estimates; and (3) lack of clear interpretability and effective utilization of uncertainties for downstream tasks. We tackle these issues through the introduction of an uncertainty-aware IoU-based loss, an evidence-aware multi-task loss function, and the implementation of a post-processing stage for uncertainty refinement. Our experimental results demonstrate that probabilistic detectors trained using the outputs of MEDL-U surpass deterministic detectors trained using outputs from previous 3D annotators on the KITTI val set for all difficulty levels. Moreover, MEDL-U achieves state-of-the-art results on the KITTI official test set compared to existing 3D automatic annotators.

Figures (13)

• Figure 1: Illustration of the proposed MEDL-U in comparison with current state-of-the-art 3D autolabeler, MTrans Liu2022MultimodalTF. MEDL-U not only generates pseudo labels but also estimates the associated uncertainties to indicate the inaccuracy of the pseudo labels. Ground-truth boxes and pseudo labels are colored red and blue, respectively.
• Figure 2: Architecture of the Training and Automatic Annotation Workflow of MEDL-U. The evidential box head regresses the evidential parameters which can be used to calculate the 3D box parameters and the uncertainties. During the automatic annotation, MEDL-U regresses 3D box parameters and the associated uncertainties for the unlabeled data. In the downstream training of probabilistic 3D detectors, the generated pseudo labels provide supervision during training and the associated box parameter uncertainties serve as factors for reweighting via the KLD loss.
• Figure C.1: Visualizing the mean epistemic uncertainties for each 3D box parameter during training when the model only uses the evidential loss with uncertainty calibration term Amini2019DeepER to supervise the training of the 3D autolabeler.
• Figure C.2: Visualizing the mean epistemic uncertainties for each 3D box parameter during training when the model includes the evidence-aware multi-task loss to supervise the training of the 3D autolabeler.
• Figure D.3: Visualizing the effect on IoU of prediction and the ground truth after incorporating the uncertainty-aware IoU loss on the model during training on the KITTI val set.