Uncertainty Estimation for 3D Object Detection via Evidential Learning

TL;DR

A framework for quantifying uncertainty in 3D object detection is introduced by leveraging an evidential learning loss on Bird's Eye View representations in the 3D detector and demonstrating both the efficacy and importance of these uncertainty estimates on identifying out-of-distribution scenes, poorly localized objects, and missing (false negative) detections.

Abstract

3D object detection is an essential task for computer vision applications in autonomous vehicles and robotics. However, models often struggle to quantify detection reliability, leading to poor performance on unfamiliar scenes. We introduce a framework for quantifying uncertainty in 3D object detection by leveraging an evidential learning loss on Bird's Eye View representations in the 3D detector. These uncertainty estimates require minimal computational overhead and are generalizable across different architectures. We demonstrate both the efficacy and importance of these uncertainty estimates on identifying out-of-distribution scenes, poorly localized objects, and missing (false negative) detections; our framework consistently improves over baselines by 10-20% on average. Finally, we integrate this suite of tasks into a system where a 3D object detector auto-labels driving scenes and our uncertainty estimates verify label correctness before the labels are used to train a second model. Here, our uncertainty-driven verification results in a 1% improvement in mAP and a 1-2% improvement in NDS.

Figures (5)

• Figure 1: 3D Object Detection Uncertainty Estimation Framework. Our Evidential Deep Learning approach jointly generates heatmap probabilities for objects within Bird's Eye View and their corresponding uncertainty values, which allows us to detect several critical problems within autonomous driving, namely (left) identifying out-of-distribution scenes (e.g., with bad weather conditions), (middle) erroneous predicted boxes, and (right) missed objects (e.g., missed grey and white cars in the image). The uncertainty estimates guide selective human verification, leading to improvements in detection metrics (e.g., mean Average Precision (mAP) and nuScenes Detection Score (NDS)).
• Figure 2: Model architecture with EDL Heatmap Head. We replace the standard heatmap head with an Evidential Deep Learning (EDL) head, which predicts both object presence probabilities and uncertainty by outputting $\boldsymbol{\alpha}_{i}$ and $\boldsymbol{\beta}_{i}$ for each BEV cell.
• Figure 3: Uncertainty at different levels.(a) Scene-level uncertainty aggregates uncertainty values across all BEV cells in a scene to produce an overall uncertainty score, which help detect OOD scenes. (b) Box-level uncertainty focuses on each predicted bounding box's uncertainty using ROI pooling, allowing for the identification of poorly localized bounding boxes.
• Figure 4: Scene out-of-distribution detection ROC and PR curves evaluation. ROC and PR curves for the OOD detection task using the uncertainty measure described in Section \ref{['ssec:ood_detection']}. A higher position of the curve indicates a better ability of the uncertainty measure to detect OOD scenes. Our uncertainty measure outperforms other methods by a significant margin across various setups.
• Figure 5: ROC and PR curve evaluation for the detection of erroneous boxes. ROC and PR curves for the erroneous box detection task using the uncertainty measure in Section \ref{['ssec:box_unc']}. A higher position of the curve indicates a better ability of the uncertainty measure to detect erroneous boxes predicted by the model. Our uncertainty measure outperforms baselines across various setups.