Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

VIO-DualProNet: Visual-Inertial Odometry with Learning Based Process Noise Covariance

Dan Solodar, Itzik Klein

TL;DR

This work tackles the sensitivity of visual-inertial odometry to inertial noise variance by introducing VIO-DualProNet, which online regresses inertial noise covariances from IMU data and feeds them into a VINS-Mono based optimization. The DualProNet network comprises Accel-ProNet and Gyro-ProNet, trained on Euroc-MAV data to predict axis-wise noise variances, and is integrated into a factor-graph VIO framework with IMU pre-integration and visual reprojection factors. Results show open-loop RMSE for noise covariance estimation is low, and closed-loop VIO with DualProNet achieves significant ATE improvements (average ~25%), outperforming constant-covariance baselines in most sequences and demonstrating robustness across changing inertial conditions. Collectively, the approach reduces the need for manual covariance tuning and enhances localization accuracy for VIO-dependent applications, with potential for broader use in estimation problems involving inertial sensors.

Abstract

Visual-inertial odometry (VIO) is a vital technique used in robotics, augmented reality, and autonomous vehicles. It combines visual and inertial measurements to accurately estimate position and orientation. Existing VIO methods assume a fixed noise covariance for the inertial uncertainty. However, accurately determining in real-time the noise variance of the inertial sensors presents a significant challenge as the uncertainty changes throughout the operation leading to suboptimal performance and reduced accuracy. To circumvent this, we propose VIO-DualProNet, a novel approach that utilizes deep learning methods to dynamically estimate the inertial noise uncertainty in real-time. By designing and training a deep neural network to predict inertial noise uncertainty using only inertial sensor measurements, and integrating it into the VINS-Mono algorithm, we demonstrate a substantial improvement in accuracy and robustness, enhancing VIO performance and potentially benefiting other VIO-based systems for precise localization and mapping across diverse conditions.

VIO-DualProNet: Visual-Inertial Odometry with Learning Based Process Noise Covariance

TL;DR

This work tackles the sensitivity of visual-inertial odometry to inertial noise variance by introducing VIO-DualProNet, which online regresses inertial noise covariances from IMU data and feeds them into a VINS-Mono based optimization. The DualProNet network comprises Accel-ProNet and Gyro-ProNet, trained on Euroc-MAV data to predict axis-wise noise variances, and is integrated into a factor-graph VIO framework with IMU pre-integration and visual reprojection factors. Results show open-loop RMSE for noise covariance estimation is low, and closed-loop VIO with DualProNet achieves significant ATE improvements (average ~25%), outperforming constant-covariance baselines in most sequences and demonstrating robustness across changing inertial conditions. Collectively, the approach reduces the need for manual covariance tuning and enhances localization accuracy for VIO-dependent applications, with potential for broader use in estimation problems involving inertial sensors.

Abstract

Visual-inertial odometry (VIO) is a vital technique used in robotics, augmented reality, and autonomous vehicles. It combines visual and inertial measurements to accurately estimate position and orientation. Existing VIO methods assume a fixed noise covariance for the inertial uncertainty. However, accurately determining in real-time the noise variance of the inertial sensors presents a significant challenge as the uncertainty changes throughout the operation leading to suboptimal performance and reduced accuracy. To circumvent this, we propose VIO-DualProNet, a novel approach that utilizes deep learning methods to dynamically estimate the inertial noise uncertainty in real-time. By designing and training a deep neural network to predict inertial noise uncertainty using only inertial sensor measurements, and integrating it into the VINS-Mono algorithm, we demonstrate a substantial improvement in accuracy and robustness, enhancing VIO performance and potentially benefiting other VIO-based systems for precise localization and mapping across diverse conditions.
Paper Structure (21 sections, 35 equations, 5 figures, 3 tables)

This paper contains 21 sections, 35 equations, 5 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Visual-Inertial Odometry (VIO)
  4. Adaptive Noise Estimation
  5. Deep Learning for SLAM and VIO
  6. Proposed Approach
  7. VIO Factor Graph Optimization
  8. IMU Pre-integration Factor
  9. Visual Reprojection Factor
  10. DualProNet
  11. Model Training and Fitting
  12. Dataset
  13. Data Preprocessing
  14. Model Training
  15. Experimental Results
  16. ...and 6 more sections

Figures (5)

  • Figure 1: VIO-DualProNet architecture. The baseline VIO algorithm is presented with our DualProNet addition colored in green.
  • Figure 2: The sliding window process in VIO algorithms, including IMU pre-integration factors between consecutive frames and visual features reprojection factors.
  • Figure 3: Network architecture for regressing the inertial sensor measurement uncertainty.
  • Figure 4: Proposed DualProNet architecture, combining the Accel-ProNet model for regressing the accelerometer readings uncertainty and the Gyro-ProNet model for regressing the gyroscope readings uncertainty.
  • Figure 5: Euroc-MAV trajectories including the ground truth trajectories and the estimated trajectories from VINS-Mono using the original constant noise covariance values compared to our proposed VIO-DualProNet.