Uncertainty Estimation and Out-of-Distribution Detection for LiDAR Scene Semantic Segmentation

TL;DR

A method to distinguish in-distribution from OOD samples and quantify both epistemic and aleatoric uncertainties using the feature space of a single deterministic model is proposed, demonstrating its superior performance in real-world applications for quantifying epistemic and aleatoric uncertainty, as well as detecting OOD samples.

Abstract

Safe navigation in new environments requires autonomous vehicles and robots to accurately interpret their surroundings, relying on LiDAR scene segmentation, out-of-distribution (OOD) obstacle detection, and uncertainty computation. We propose a method to distinguish in-distribution (ID) from OOD samples and quantify both epistemic and aleatoric uncertainties using the feature space of a single deterministic model. After training a semantic segmentation network, a Gaussian Mixture Model (GMM) is fitted to its feature space. OOD samples are detected by checking if their squared Mahalanobis distances to each Gaussian component conform to a chi-squared distribution, eliminating the need for an additional OOD training set. Given that the estimated mean and covariance matrix of a multivariate Gaussian distribution follow Gaussian and Inverse-Wishart distributions, multiple GMMs are generated by sampling from these distributions to assess epistemic uncertainty through classification variability. Aleatoric uncertainty is derived from the entropy of responsibility values within Gaussian components. Comparing our method with deep ensembles and logit-sampling for uncertainty computation demonstrates its superior performance in real-world applications for quantifying epistemic and aleatoric uncertainty, as well as detecting OOD samples. While deep ensembles miss some highly uncertain samples, our method successfully detects them and assigns high epistemic uncertainty.

Figures (6)

• Figure 1: Overview of our proposed method for OOD detection and epistemic and aleatoric uncertainty estimation. In this work, SalsaNext cortinhal2020 is used as U-Net shaped model, but other models designed for semantic segmentation could be used as well. The outputs of the layer colored in yellow are selected as the feature space to which a GMM is fitted and the layer colored in purple represents the classification layer producing the softmax outputs.
• Figure 2: OOD detection using our proposed method across two different scans. From top to bottom for each (a) and (b): ground truth map, with OOD samples marked in black and OOD prediction map, illustrating OOD samples in red and ID samples in blue color.
• Figure 3: Epistemic and aleatoric uncertainty quantification by our proposed method and the DDU approach. From top to bottom: Ground truth map, predicted semantic labels, our proposed epistemic uncertainty map derived from the entropy of classification outputs across sampled GMMs, Epistemic map from DDU derived from the probability density of a single GMM with empirical means and covariance matrices, Our proposed aleatoric uncertainty map by the entropy of responsibility values, aleatoric map from DDU by the entropy of softmax outputs, and the Error map showing erroneous predictions in black. The highest epistemic uncertainty values are observed in OOD samples and misclassified objects, as highlighted in the red boxes around parts of the bicyclist, the ground beneath the car, the misclassified sidewalk, and the often-confused pole and trunk. The highest aleatoric uncertainty values are observed along class borders and in distant objects.
• Figure 4: Comparison of epistemic uncertainties computed by our proposed method, DDU approach and deep ensembles. From top to bottom: ground truth map, predicted semantic labels, OOD prediction map by our method, eepistemic map by our method, epistemic map by DDU, epistemic map by deep ensembles, and the Error map. OOD samples are shown in red in the OOD prediction map and in black in the Ground truth map. Red boxes highlight the uncertain samples that our proposed method successfully detected, which were missed by the deep ensembles and DDU approaches.
• Figure 5: Comparison of aleatoric uncertainty computed by (a) our proposed method and (b) logit-sampling. The figure displays, from top to bottom: ground truth map, predicted map, aleatoric uncertainty, and error map. In (a), the first aleatoric map shows our proposed method, and the second shows the DDU approach. The red box highlights the limitations of logit-sampling, which misclassifies the terrain, leading to high errors and elevated aleatoric uncertainty. In contrast, our method demonstrates improved classification and more accurate class distinction. Comparing the aleatoric maps in (a) shows that our method aligns closely with the error map, while the DDU approach fails to detect high uncertainty in the trunk area highlighted in the red box.