VINS-Multi: A Robust Asynchronous Multi-camera-IMU State Estimator

TL;DR

VINS-Multi addresses robust state estimation for asynchronous multi-camera-IMU systems by combining parallel per-camera front ends, a front-end coordinator with dynamic feature allocation and a frame-priority strategy, and a sliding-window back-end that fuses IMU pre-integration with multi-view visual data, delivering rate-ordered odometries at $30$ and $500$ Hz. The approach supports mixed camera types and asynchronous inputs, with a feature-based marginalization strategy to handle uneven frame arrivals and camera failures. Key contributions include the dynamic per-camera feature budgeting, frame-priority coordination, and a robust back-end that extends prior VINS frameworks to multi-camera asynchronous setups, all validated on a quadrotor in challenging scenarios. Results show improved accuracy and robustness over single-camera baselines and ablations, highlighting practical impact for field robotics with limited hardware synchronization.

Abstract

State estimation is a critical foundational module in robotics applications, where robustness and performance are paramount. Although in recent years, many works have been focusing on improving one of the most widely adopted state estimation methods, visual inertial odometry (VIO), by incorporating multiple cameras, these efforts predominantly address synchronous camera systems. Asynchronous cameras, which offer simpler hardware configurations and enhanced resilience, have been largely overlooked. To fill this gap, this paper presents VINS-Multi, a novel multi-camera-IMU state estimator for asynchronous cameras. The estimator comprises parallel front ends, a front end coordinator, and a back end optimization module capable of handling asynchronous input frames. It utilizes the frames effectively through a dynamic feature number allocation and a frame priority coordination strategy. The proposed estimator is integrated into a customized quadrotor platform and tested in multiple realistic and challenging scenarios to validate its practicality. Additionally, comprehensive benchmark results are provided to showcase the robustness and superior performance of the proposed estimator.

Figures (8)

• Figure 1: The deployment of VINS-Multi in aerial vent pipe inspection scenarios. The green lines are the estimated trajectory and the white arrows indicate the estimated odometries. The maps are constructed during the flights and the color code indicates the height.
• Figure 2: The system architecture of the proposed state estimator. Note that the camera modules can be of mixed types.
• Figure 3: A schematic of the sliding window incorporating IMU data and visual (including depth) data for optimization.
• Figure 4: An illustration of the marginalization strategy.
• Figure 5: The quadrotor deployed in experiments. The system is compatible with mixed types of monocular, RGBD, and stereo camera modules.