GLENet: Boosting 3D Object Detectors with Generative Label Uncertainty Estimation

TL;DR

GLENet reframes 3D label uncertainty as the diversity of plausible ground-truth bounding boxes and uses a conditional variational autoencoder to model $p(X\vert C)$ via a latent variable $z$, enabling multiple plausible box samples and quantified label uncertainty for supervision. The method provides a plug-and-play module that feeds uncertainty into probabilistic detectors, supplemented by an Uncertainty-aware Quality Estimator (UAQE) and 3D Variance Voting to refine localization confidence and final boxes. Empirical results on KITTI and Waymo show consistent improvements across base detectors, with GLENet-VR achieving state-of-the-art single-modal performance on KITTI and strong gains at long ranges in Waymo. The work advances practical 3D detection by incorporating data-driven label ambiguity into training, regularizing localization and yielding more reliable, uncertainty-aware detections.

Abstract

The inherent ambiguity in ground-truth annotations of 3D bounding boxes, caused by occlusions, signal missing, or manual annotation errors, can confuse deep 3D object detectors during training, thus deteriorating detection accuracy. However, existing methods overlook such issues to some extent and treat the labels as deterministic. In this paper, we formulate the label uncertainty problem as the diversity of potentially plausible bounding boxes of objects. Then, we propose GLENet, a generative framework adapted from conditional variational autoencoders, to model the one-to-many relationship between a typical 3D object and its potential ground-truth bounding boxes with latent variables. The label uncertainty generated by GLENet is a plug-and-play module and can be conveniently integrated into existing deep 3D detectors to build probabilistic detectors and supervise the learning of the localization uncertainty. Besides, we propose an uncertainty-aware quality estimator architecture in probabilistic detectors to guide the training of the IoU-branch with predicted localization uncertainty. We incorporate the proposed methods into various popular base 3D detectors and demonstrate significant and consistent performance gains on both KITTI and Waymo benchmark datasets. Especially, the proposed GLENet-VR outperforms all published LiDAR-based approaches by a large margin and achieves the top rank among single-modal methods on the challenging KITTI test set. The source code and pre-trained models are publicly available at \url{https://github.com/Eaphan/GLENet}.

Figures (12)

• Figure 1: (a) Given an object with an incomplete LiDAR observation, there may exist multiple potentially plausible ground-truth bounding boxes with varying sizes and shapes. (b) Ambiguity and inaccuracy can be inevitable in the labeling process when annotations are derived from 2D images and partial points. In the given cases, similar point clouds of the car category with only the rear part can be annotated with different ground-truth boxes of varying lengths.
• Figure 2: Illustration of two different learning paradigms of probabilistic object detectors. (a) Methods that adopt probabilistic modeling in the detection head but essentially still ignore the issue of ambiguity in ground-truth bounding boxes. (b) Methods that explicitly estimate ground-truth bounding box distributions to be used as more reliable supervision signals.
• Figure 3: Illustration of multiple potentially plausible bounding boxes from GLENet on the KITTI dataset by sampling latent variables multiple times. The point cloud, annotated ground-truth boxes, and predictions of GLENet are colored in black, red, and green, respectively. GLENet produces diverse predictions for objects represented with sparse point clouds and incomplete outlines, and consistent bounding boxes for objects with high-quality point clouds. The variance of the multiple predictions by GLENet is used to estimate the uncertainty of the annotated ground-truth bounding boxes.
• Figure 4: The overall workflow of GLENet. In the training phase, we learn parameters $\mu$ and $\sigma$ (resp. $\mu^{\prime}$ and $\sigma^{\prime}$ ) of latent variable $z$ (resp. $z^{\prime}$) through the prior network (resp. recognition network), after which a sample of $z^{\prime}$ and the corresponding geometrical embedding produced by the context encoder are jointly exploited to estimate the bounding box distribution. In the inference phase, we sample from the distribution of $z$ multiple times to generate different bounding boxes, whose variance we use as label uncertainty. Note we denote multiple sampling with black, orange, and green lines in subgraph (a).
• Figure 5: Illustration of the KL-divergence between distributions as a function of localization error $|y_g-\hat{y}|$ and estimated localization variance $\hat{\sigma}$ given different label uncertainty $\sigma$. With label uncertainty $\sigma$ estimated by GLENet instead of zero, the gradient is smoother when the loss converges to the minimum. Besides, the $L_{reg}$ is smaller when $\sigma$ is larger, which prevents the model from overfitting to uncertain annotations.