Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MAVIS: Multi-Camera Augmented Visual-Inertial SLAM using SE2(3) Based Exact IMU Pre-integration

Yifu Wang, Yonhon Ng, Inkyu Sa, Alvaro Parra, Cristian Rodriguez, Tao Jun Lin, Hongdong Li

TL;DR

This work tackles the challenge of robust, accurate visual-inertial SLAM for multi-camera systems with partial overlap. It advances the field by introducing an exact IMU pre-integration formulation based on the exponential map of the automorphism of $SE_2(3)$, enabling precise motion propagation during fast rotations and long integration times. The authors extend front-end tracking and back-end optimization to handle multiple synchronized cameras, along with multi-camera visual pre-processing, robust initialization, loop closure, and map fusion across sessions. Extensive experiments on EuRoC and Hilti SLAM Challenge 2023 demonstrate state-of-the-art accuracy and real-time CPU performance, highlighting the method’s practical value for real-world robotics and AR/VR applications.

Abstract

We present a novel optimization-based Visual-Inertial SLAM system designed for multiple partially overlapped camera systems, named MAVIS. Our framework fully exploits the benefits of wide field-of-view from multi-camera systems, and the metric scale measurements provided by an inertial measurement unit (IMU). We introduce an improved IMU pre-integration formulation based on the exponential function of an automorphism of SE_2(3), which can effectively enhance tracking performance under fast rotational motion and extended integration time. Furthermore, we extend conventional front-end tracking and back-end optimization module designed for monocular or stereo setup towards multi-camera systems, and introduce implementation details that contribute to the performance of our system in challenging scenarios. The practical validity of our approach is supported by our experiments on public datasets. Our MAVIS won the first place in all the vision-IMU tracks (single and multi-session SLAM) on Hilti SLAM Challenge 2023 with 1.7 times the score compared to the second place.

MAVIS: Multi-Camera Augmented Visual-Inertial SLAM using SE2(3) Based Exact IMU Pre-integration

TL;DR

This work tackles the challenge of robust, accurate visual-inertial SLAM for multi-camera systems with partial overlap. It advances the field by introducing an exact IMU pre-integration formulation based on the exponential map of the automorphism of , enabling precise motion propagation during fast rotations and long integration times. The authors extend front-end tracking and back-end optimization to handle multiple synchronized cameras, along with multi-camera visual pre-processing, robust initialization, loop closure, and map fusion across sessions. Extensive experiments on EuRoC and Hilti SLAM Challenge 2023 demonstrate state-of-the-art accuracy and real-time CPU performance, highlighting the method’s practical value for real-world robotics and AR/VR applications.

Abstract

We present a novel optimization-based Visual-Inertial SLAM system designed for multiple partially overlapped camera systems, named MAVIS. Our framework fully exploits the benefits of wide field-of-view from multi-camera systems, and the metric scale measurements provided by an inertial measurement unit (IMU). We introduce an improved IMU pre-integration formulation based on the exponential function of an automorphism of SE_2(3), which can effectively enhance tracking performance under fast rotational motion and extended integration time. Furthermore, we extend conventional front-end tracking and back-end optimization module designed for monocular or stereo setup towards multi-camera systems, and introduce implementation details that contribute to the performance of our system in challenging scenarios. The practical validity of our approach is supported by our experiments on public datasets. Our MAVIS won the first place in all the vision-IMU tracks (single and multi-session SLAM) on Hilti SLAM Challenge 2023 with 1.7 times the score compared to the second place.
Paper Structure (17 sections, 19 equations, 5 figures, 2 tables)

This paper contains 17 sections, 19 equations, 5 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Inertial Preintegration
  4. Methodology
  5. IMU intrinsic compensation
  6. IMU pre-integration
  7. Front-end tracking
  8. Back-end optimization
  9. Application to Multi-camera VI-SLAM
  10. Visual measurement pre-processing
  11. Camera-IMU initialization
  12. Loop closure
  13. Multi-Agent SLAM
  14. Experiments
  15. Performance on EuRoC datasets
  16. ...and 2 more sections

Figures (5)

  • Figure 1: AlphaSense multi-camera module as an example of multi-camera system analyzed in this paper, with a forward-facing stereo cameras and multiple sideward monocular cameras.
  • Figure 2: Overview of our proposed visual-inertial localization and mapping pipeline for multi-camera systems.
  • Figure 3: Geometric relationships of local feature matching across multiple cameras.
  • Figure 4: Multiple detected closed loops from site1_handheld_4 sequence in Hilti SLAM Challenge 2023. Accumulated drifts are significantly reduced, ensuring enhanced accuracy and robustness.
  • Figure 5: Perspective view of reconstructed scenes. From left to right: site_1, site_2, site_3. Trajectory is coloured in sequence from red, green, blue, magenta.