Uncertainty-Aware Online Extrinsic Calibration: A Conformal Prediction Approach

TL;DR

This work tackles the problem of quantifying and leveraging uncertainty in online extrinsic calibration for multi-sensor fusion. It proposes a pipeline that combines Monte Carlo Dropout for model uncertainty with Conformal Prediction to produce prediction intervals that guarantee coverage at a specified level, while remaining compatible with existing calibration models. Key contributions include a CP-based framework tailored to online extrinsic calibration, a practical integration with deep learning backbones, and extensive validation on KITTI and DSEC across RGB-LiDAR and Event-LiDAR modalities, along with ablations that highlight the necessity of CP over MCD alone and the impact of sensor resolution. The approach enhances robustness and interpretability of calibration in dynamic environments, enabling decisions such as triggering recalibration or adjusting sensor-fusion confidence, which is critical for safe and reliable autonomous systems.

Abstract

Accurate sensor calibration is crucial for autonomous systems, yet its uncertainty quantification remains underexplored. We present the first approach to integrate uncertainty awareness into online extrinsic calibration, combining Monte Carlo Dropout with Conformal Prediction to generate prediction intervals with a guaranteed level of coverage. Our method proposes a framework to enhance existing calibration models with uncertainty quantification, compatible with various network architectures. Validated on KITTI (RGB Camera-LiDAR) and DSEC (Event Camera-LiDAR) datasets, we demonstrate effectiveness across different visual sensor types, measuring performance with adapted metrics to evaluate the efficiency and reliability of the intervals. By providing calibration parameters with quantifiable confidence measures, we offer insights into the reliability of calibration estimates, which can greatly improve the robustness of sensor fusion in dynamic environments and usefully serve the Computer Vision community.

Figures (6)

• Figure 1: Overview of the uncertainty-aware calibration pipeline. (Left) The deep learning Calibration Network estimates parameters from sensor data. (Center) MCD is applied to generate multiple predictions, producing a mean estimate $\hat{y}_\text{mean}$ (the prediction) and a standard deviation $\sigma$ (measuring the uncertainty). (Right) The CP method is beforehand calibrated on a separate uncertainty calibration set, then can be used to estimate intervals with a $1-\alpha$ coverage.
• Figure 2: Axes of rotation and translation of the spatial transformation in the LiDAR frame.
• Figure 3: LiDAR point cloud projections onto an image frame from KITTI geigerVisionMeetsRobotics2013 under varying Pitch calibration. (Top) Ground truth Pitch minus 0.25°. (Middle): Ground truth Pitch. (Bottom) Ground truth Pitch plus 0.25°. This ±0.25° range represents an extreme case of our 90% confidence interval for Pitch. The visual differences in projections are minimal, demonstrating the high precision and practical robustness of our calibration method in challenging scenarios.
• Figure 4: Calibration curves for extrinsic parameters on the KITTI geigerVisionMeetsRobotics2013 and DSEC gehrigDSECStereoEvent2021 datasets, showing observed versus expected coverage for each degree of freedom. Better seen on screen.
• Figure 5: Ordered prediction interval plots for the six degrees of freedom (in translation and rotation) on the KITTI geigerVisionMeetsRobotics2013 dataset. The overlaid shaded regions represent the intervals corresponding to expected coverage levels of 90%, 95%, and 99%. The ground truth values should fall within the respective intervals for at least the specified proportion of samples. The X-axis represents the ordered test samples sorted by the ground truth values for each degree of freedom, while the Y-axis indicates the deviation from the ground truth. All curves are smoothed using a moving average to enhance readability.