Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Tightly-Coupled Radar-Visual-Inertial Odometry

Morten Nissov, Mohit Singh, Kostas Alexis

Abstract

Visual-Inertial Odometry (VIO) is a staple for reliable state estimation on constrained and lightweight platforms due to its versatility and demonstrated performance. However, pertinent challenges regarding robust operation in dark, low-texture, obscured environments complicate the use of such methods. Alternatively, Frequency Modulated Continuous Wave (FMCW) radars, and by extension Radar-Inertial Odometry (RIO), offer robustness to these visual challenges, albeit at the cost of reduced information density and worse long-term accuracy. To address these limitations, this work combines the two in a tightly coupled manner, enabling the resulting method to operate robustly regardless of environmental conditions or trajectory dynamics. The proposed method fuses image features, radar Doppler measurements, and Inertial Measurement Unit (IMU) measurements within an Iterated Extended Kalman Filter (IEKF) in real-time, with radar range data augmenting the visual feature depth initialization. The method is evaluated through flight experiments conducted in both indoor and outdoor environments, as well as through challenges to both exteroceptive modalities (such as darkness, fog, or fast flight), thoroughly demonstrating its robustness. The implementation of the proposed method is available at: https://github.com/ntnu-arl/radvio.

Tightly-Coupled Radar-Visual-Inertial Odometry

Abstract

Visual-Inertial Odometry (VIO) is a staple for reliable state estimation on constrained and lightweight platforms due to its versatility and demonstrated performance. However, pertinent challenges regarding robust operation in dark, low-texture, obscured environments complicate the use of such methods. Alternatively, Frequency Modulated Continuous Wave (FMCW) radars, and by extension Radar-Inertial Odometry (RIO), offer robustness to these visual challenges, albeit at the cost of reduced information density and worse long-term accuracy. To address these limitations, this work combines the two in a tightly coupled manner, enabling the resulting method to operate robustly regardless of environmental conditions or trajectory dynamics. The proposed method fuses image features, radar Doppler measurements, and Inertial Measurement Unit (IMU) measurements within an Iterated Extended Kalman Filter (IEKF) in real-time, with radar range data augmenting the visual feature depth initialization. The method is evaluated through flight experiments conducted in both indoor and outdoor environments, as well as through challenges to both exteroceptive modalities (such as darkness, fog, or fast flight), thoroughly demonstrating its robustness. The implementation of the proposed method is available at: https://github.com/ntnu-arl/radvio.
Paper Structure (15 sections, 12 equations, 9 figures, 2 tables)

This paper contains 15 sections, 12 equations, 9 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Method
  4. Notation and Coordinate Frames
  5. Basics of fmcw Radar
  6. Aided Inertial Navigation
  7. Radial Speed Measurement Update
  8. Feature Depth Initialization
  9. Evaluation
  10. Nominal Conditions
  11. Challenging Environments for Vision
  12. Darkness
  13. Fog
  14. Challenging Environments for Radar
  15. Conclusions

Figures (9)

  • Figure 1: The aerial platform used to evaluate the proposed method, shown flying in nominal, obscured (due to fog), and dark environments.
  • Figure 2: Information flow of the proposed method from the imu, radar, and camera sensors to the odometry and local-map outputs.
  • Figure 3: Correlation between the feature patch in the image and the voxels resulting from querying the feature depth.
  • Figure 4: Position estimates of the proposed method alongside ablations from forest4. The image shows the good visual conditions of the Forest environment.
  • Figure 5: Radar extrinsic estimation of RadVIO in forest1 with nominal and perturbed initial guesses. The perturbed initial guess is generated by applying a rotation of 80° about the radar sensor's $y$ axis to the nominal prior.
  • ...and 4 more figures