Omni-LIVO: Robust RGB-Colored Multi-Camera Visual-Inertial-LiDAR Odometry via Photometric Migration and ESIKF Fusion

TL;DR

Omni-LIVO tackles the robustness gap in LIVO by introducing a tightly coupled multi-camera system that leverages LiDAR geometry across extended FoVs. It integrates cross-view direct photometric alignment with an adaptive Error-State Iterated Kalman Filter to fuse LiDAR, multiple RGB cameras, and IMU data, all within a unified voxel map. The method demonstrates substantial accuracy gains and richer RGB-colored mapping across diverse environments, validated on public benchmarks and a custom dataset. The work exhibits practical viability with real-time performance on embedded hardware and highlights the benefits of multi-view constraints for robustness and colorization.

Abstract

Wide field-of-view (FoV) LiDAR sensors provide dense geometry across large environments, but existing LiDAR-inertial-visual odometry (LIVO) systems generally rely on a single camera, limiting their ability to fully exploit LiDAR-derived depth for photometric alignment and scene colorization. We present Omni-LIVO, a tightly coupled multi-camera LIVO system that leverages multi-view observations to comprehensively utilize LiDAR geometric information across extended spatial regions. Omni-LIVO introduces a Cross-View direct alignment strategy that maintains photometric consistency across non-overlapping views, and extends the Error-State Iterated Kalman Filter (ESIKF) with multi-view updates and adaptive covariance. The system is evaluated on public benchmarks and our custom dataset, showing improved accuracy and robustness over state-of-the-art LIVO, LIO, and visual-inertial SLAM baselines. Code and dataset will be released upon publication.

Figures (8)

• Figure 1: Overview of the Omni-LIVO architecture. Light yellow, light blue, and light green modules represent visual, LiDAR, and IMU data respectively. Orange borders indicate extensions to baseline methods; blue borders indicate novel contributions.
• Figure 2: Multi-sensor temporal alignment. LiDAR points are associated with the nearest camera frame, while IMU measurements provide continuous motion priors for temporally consistent fusion.
• Figure 3: Cross-View temporal migration. At $t_1$, map points $\mathbf{p}_1$, $\mathbf{p}_2$, $\mathbf{p}_3$ are observed by Left ($C_L$), Front ($C_F$), and Right ($C_R$) cameras respectively. As the platform moves, points migrate between camera views ($\mathbf{p}_2$ migrates from $C_F$ to $C_L$ at $t_2$), maintaining photometric continuity across non-overlapping $90^\circ$ FoVs through Cross-View residuals.
• Figure 4: Photometric patch associations during turning: intra-camera patches (green) remain within the same view, while migrated patches (red) form Cross-View associations across different cameras.
• Figure 5: Iterative Multi-Camera ESIKF Framework. The pipeline performs Synchronize , IMU Propagate , LiDAR Update, Select Patches, and Adaptive Cov, followed by Init VIO and iterative updates of the photometric VIO loop until convergence.