Probabilistic Uncertainty Quantification of Prediction Models with Application to Visual Localization

TL;DR

This paper addresses the challenge of obtaining calibrated uncertainty from neural-network predictions in visual localization for autonomous driving. It introduces a sensor error model that maps an internal predictor signal, namely the number of keypoint matches, to a probabilistic uncertainty suitable for integration with Kalman-like estimators, without retraining or altering the network. The authors develop both a single Gaussian and a Gaussian Mixture representation, with the latter shown to better capture multi-modal error behavior, and demonstrate integration via Sigma Point and Gaussian Sum Filters. Experiments on Ithaca365 indicate well-calibrated uncertainties and improved localization performance, even under adversarial conditions, with strong generalization to unseen locations. The approach provides a practical pathway to robust probabilistic perception in real-world robotics, enabling more reliable planning and control without costly model redesigns.

Abstract

The uncertainty quantification of prediction models (e.g., neural networks) is crucial for their adoption in many robotics applications. This is arguably as important as making accurate predictions, especially for safety-critical applications such as self-driving cars. This paper proposes our approach to uncertainty quantification in the context of visual localization for autonomous driving, where we predict locations from images. Our proposed framework estimates probabilistic uncertainty by creating a sensor error model that maps an internal output of the prediction model to the uncertainty. The sensor error model is created using multiple image databases of visual localization, each with ground-truth location. We demonstrate the accuracy of our uncertainty prediction framework using the Ithaca365 dataset, which includes variations in lighting, weather (sunny, snowy, night), and alignment errors between databases. We analyze both the predicted uncertainty and its incorporation into a Kalman-based localization filter. Our results show that prediction error variations increase with poor weather and lighting condition, leading to greater uncertainty and outliers, which can be predicted by our proposed uncertainty model. Additionally, our probabilistic error model enables the filter to remove ad hoc sensor gating, as the uncertainty automatically adjusts the model to the input data

Figures (9)

• Figure 1: Pipeline for location prediction form image-retrieval using multiple traversals.
• Figure 2: GPS locations of several traversals (zoomed in for illustration; full trajectory is not shown). Using multiple traversals increases the chances that a database image is closer to the query image location (i.e., smaller theoretical error).
• Figure 3: Relationship between number of keypoint matches and location error for two different database traversals.
• Figure 4: Pipeline for uncertainty prediction. Top: creating sensor error model. Bottom: using sensor error model in inference.
• Figure 5: Reliability diagrams of Gaussian sensor error model for $Q^\text{sunny}$, $Q^\text{night}$, and $Q^\text{snowy}$.