Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LiDAR-enhanced 3D Gaussian Splatting Mapping

Jian Shen, Huai Yu, Ji Wu, Wen Yang, Gui-Song Xia

TL;DR

LiGSM addresses the fragility of purely visual 3D Gaussian Splatting (3DGS) mappings by integrating LiDAR geometry to improve initialization and pose estimation. It fuses LiDAR point clouds with monocular images through joint optimization of camera-LiDAR extrinsics and poses, and uses LiDAR-driven depth supervision to enhance rendering fidelity. The approach includes LiDAR-based initialization of the 3DGS, a cycle-consistent global pose graph, and a dual supervision scheme combining color and depth for map optimization. Experiments on public datasets and a self-collected sensor pod demonstrate improved pose tracking and rendering quality, illustrating robustness to sensor miscalibration and scene variation.

Abstract

This paper introduces LiGSM, a novel LiDAR-enhanced 3D Gaussian Splatting (3DGS) mapping framework that improves the accuracy and robustness of 3D scene mapping by integrating LiDAR data. LiGSM constructs joint loss from images and LiDAR point clouds to estimate the poses and optimize their extrinsic parameters, enabling dynamic adaptation to variations in sensor alignment. Furthermore, it leverages LiDAR point clouds to initialize 3DGS, providing a denser and more reliable starting points compared to sparse SfM points. In scene rendering, the framework augments standard image-based supervision with depth maps generated from LiDAR projections, ensuring an accurate scene representation in both geometry and photometry. Experiments on public and self-collected datasets demonstrate that LiGSM outperforms comparative methods in pose tracking and scene rendering.

LiDAR-enhanced 3D Gaussian Splatting Mapping

TL;DR

LiGSM addresses the fragility of purely visual 3D Gaussian Splatting (3DGS) mappings by integrating LiDAR geometry to improve initialization and pose estimation. It fuses LiDAR point clouds with monocular images through joint optimization of camera-LiDAR extrinsics and poses, and uses LiDAR-driven depth supervision to enhance rendering fidelity. The approach includes LiDAR-based initialization of the 3DGS, a cycle-consistent global pose graph, and a dual supervision scheme combining color and depth for map optimization. Experiments on public datasets and a self-collected sensor pod demonstrate improved pose tracking and rendering quality, illustrating robustness to sensor miscalibration and scene variation.

Abstract

This paper introduces LiGSM, a novel LiDAR-enhanced 3D Gaussian Splatting (3DGS) mapping framework that improves the accuracy and robustness of 3D scene mapping by integrating LiDAR data. LiGSM constructs joint loss from images and LiDAR point clouds to estimate the poses and optimize their extrinsic parameters, enabling dynamic adaptation to variations in sensor alignment. Furthermore, it leverages LiDAR point clouds to initialize 3DGS, providing a denser and more reliable starting points compared to sparse SfM points. In scene rendering, the framework augments standard image-based supervision with depth maps generated from LiDAR projections, ensuring an accurate scene representation in both geometry and photometry. Experiments on public and self-collected datasets demonstrate that LiGSM outperforms comparative methods in pose tracking and scene rendering.

Paper Structure

This paper contains 25 sections, 14 equations, 7 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Dense Scene Representation for SLAM
  4. Multimodal Dense 3D Reconstruction
  5. Methodology
  6. Relative Motion Estimation
  7. Global Poses Estimation
  8. Poses and Extrinsics Optimization
  9. Camera Loss
  10. LiDAR Loss
  11. Joint Loss
  12. Joint Optimization
  13. 3DGS-based Mapping
  14. Initialization
  15. Map Updating
  16. ...and 10 more sections

Figures (7)

  • Figure 1: LiGSM takes images and LiDAR point clouds as inputs, simultaneously optimizing the LiDAR-camera extrinsics and estimating the poses. The 3DGS reconstruction results is capable of rendering color images and depth maps.
  • Figure 2: The pipeline of LiGSM. Point clouds are first used for poses and LiDAR-camera extrinsic estimation, and then to initialize 3DGS and optimize the 3D scene mapping.
  • Figure 3: Sensor pod with synchronized LiDAR, RGB, and depth cameras.
  • Figure 4: Self-collected data examples.
  • Figure 5: Rendering performance on Replica.
  • ...and 2 more figures