LIR-LIVO: A Lightweight,Robust LiDAR/Vision/Inertial Odometry with Illumination-Resilient Deep Features

TL;DR

LIR-LIVO addresses the challenge of robust, real-time odometry in environments with degraded LiDAR and illumination by fusing LiDAR depth with illumination-resilient visual features in a lightweight LiDAR–inertial–visual pipeline. The method introduces sweep recombination for temporal alignment, depth association to directly obtain visual feature depths from LiDAR data, and a shallow visual frontend using SuperPoint and LightGlue, all integrated via an INS-based EKF with a two-stage measurement update. Key contributions include uniform depth distribution for feature points, a depth-augmented visual optimization that avoids full visual landmark maps, and a compact, open-source implementation that achieves SOTA accuracy on NTU-VIRAL and Hilti'22 with low computational cost. The results demonstrate improved robustness to illumination changes and motion, enabling reliable pose estimation in challenging indoor and degraded scenarios, with practical impact for robotics on resource-constrained platforms.

Abstract

In this paper, we propose LIR-LIVO, a lightweight and robust LiDAR-inertial-visual odometry system designed for challenging illumination and degraded environments. The proposed method leverages deep learning-based illumination-resilient features and LiDAR-Inertial-Visual Odometry (LIVO). By incorporating advanced techniques such as uniform depth distribution of features enabled by depth association with LiDAR point clouds and adaptive feature matching utilizing Superpoint and LightGlue, LIR-LIVO achieves state-of-the-art (SOTA) accuracy and robustness with low computational cost. Experiments are conducted on benchmark datasets, including NTU-VIRAL, Hilti'22, and R3LIVE-Dataset. The corresponding results demonstrate that our proposed method outperforms other SOTA methods on both standard and challenging datasets. Particularly, the proposed method demonstrates robust pose estimation under poor ambient lighting conditions in the Hilti'22 dataset. The code of this work is publicly accessible on GitHub to facilitate advancements in the robotics community.

Figures (6)

• Figure 1: The 3D point cloud mapping results of LIR-LIVO on the Hilti'22 sequence "Exp16 Attic to Upper Gallery 2". The sequence was collected in the indoor environment of the Sheldonian Theatre in Oxford, categorized as a "hard" difficulty level.
• Figure 2: The framework of our LIR-LIVO. The LiDAR frame timestamps are synchronized with camera frame timestamps by sweeping recombination, enabling sequential updates of LiDAR and visual measurements. The visual frontend incorporates SuperPoint and SuperGlue frameworks, with optimized depth distribution of feature points. The black points in the frame window are the selected features with depth associated.
• Figure 3: The process of sweep recombination. The original LiDAR frame is decomposed to construct a new one synchronized with camera frame.
• Figure 4: Sensitivity differences of camera rotation and translation to landmarks at varying depths. The centers of “IP1” and “IP3” coincide.
• Figure 5: The performance of proposed visual fontend. The images are sourced from Hilti's 2022 attic_to_upper_gallery, construction_upper_level, and corridor_lower_gallery datasetshilti. These datasets were collected in low-light indoor environments and exhibit significant illumination variations.