Statistical Uncertainty Learning for Robust Visual-Inertial State Estimation

TL;DR

The paper addresses the challenge of dynamic measurement reliability in visual-inertial odometry by introducing online statistical uncertainty learning guided by multi-view geometric consistency. It integrates this learning into a dual-pipeline VIO architecture (real-time PnP tracking and sliding-window BA) where BA results update observation covariances that weight the optimization. Uncertainty is propagated from 3D landmark covariances to pixel-space via Jacobians to form adaptive information matrices, and a post-optimization learning loop refines the uncertainty model iteratively. On EuRoC, the approach achieves approximately 24% translation and 42% rotation error reductions with real-time performance, and the authors release open-source code.

Abstract

A fundamental challenge in robust visual-inertial odometry (VIO) is to dynamically assess the reliability of sensor measurements. This assessment is crucial for properly weighting the contribution of each measurement to the state estimate. Conventional methods often simplify this by assuming a static, uniform uncertainty for all measurements. This heuristic, however, may be limited in its ability to capture the dynamic error characteristics inherent in real-world data. To improve this limitation, we present a statistical framework that learns measurement reliability assessment online, directly from sensor data and optimization results. Our approach leverages multi-view geometric consistency as a form of self-supervision. This enables the system to infer landmark uncertainty and adaptively weight visual measurements during optimization. We evaluated our method on the public EuRoC dataset, demonstrating improvements in tracking accuracy with average reductions of approximately 24\% in translation error and 42\% in rotation error compared to baseline methods with fixed uncertainty parameters. The resulting framework operates in real time while showing enhanced accuracy and robustness. To facilitate reproducibility and encourage further research, the source code will be made publicly available.

Figures (4)

• Figure 1: Visualization of learned uncertainty estimates for visual landmarks. The ellipses represent the projected 2D covariance of 3D landmark uncertainties onto the image plane, illustrating the spatial distribution of measurement reliability. Color coding indicates uncertainty magnitude: red ellipses correspond to lower uncertainties (higher reliability), while blue ellipses indicate higher uncertainties (lower reliability). This adaptive uncertainty modeling enables observation-specific weighting in bundle adjustment optimization.
• Figure 2: Overview of the proposed visual-inertial system architecture featuring a dual-pipeline design. The upper pipeline handles real-time pose tracking via Perspective-n-Point (PnP) optimization, while the lower pipeline performs map refinement through sliding window bundle adjustment (BA) integrated with our statistical uncertainty learning module. The learned uncertainty estimates are propagated back to the tracking pipeline, establishing a feedback mechanism that enhances system performance.
• Figure 3: Sliding window bundle adjustment framework. The system maintains a fixed-size window of keyframes while marginalizing older frames. Visual observations connect 3D landmarks to multiple keyframes, while IMU measurements provide constraints between consecutive poses. The optimization jointly estimates camera poses, landmark positions, and IMU biases within the sliding window.
• Figure 4: Quantitative evaluation of frame-to-frame relative pose accuracy across EuRoC sequences. Upper panels display translation RMSE (cm), lower panels show rotation RMSE (degrees). Our statistical uncertainty learning approach (Phase 2) demonstrates consistent improvements over baseline and geometric uncertainty methods (Phase 1) across most sequences. The progressive improvement trend (Baseline → Phase 1 → Phase 2) validates the effectiveness of adaptive uncertainty learning for visual-inertial odometry.