GeomGS: LiDAR-Guided Geometry-Aware Gaussian Splatting for Robot Localization

TL;DR

GeomGS addresses the mismatch between 3D Gaussian Splatting representations and real-world geometry by integrating LiDAR data through a Geometric Confidence Score ($GCS$) and probabilistic distance constraints. The method yields geometry-faithful renderable maps while preserving or improving image quality, and introduces a unified localization pipeline that couples Weighted ICP with image-based pose refinement on renderings from the accurate map. Key contributions include the $GCS$-driven loss terms ($\mathcal{L}_{geom}$, $\mathcal{L}_{prob}$), a LiDAR-augmented point accumulation, and a robust localization scheme leveraging both LiDAR geometry and photometric cues. Experimental results on KITTI and KITTI-360 show state-of-the-art geometric accuracy and localization performance, with notable improvements over prior 3DGS approaches and SfM baselines.

Abstract

Mapping and localization are crucial problems in robotics and autonomous driving. Recent advances in 3D Gaussian Splatting (3DGS) have enabled precise 3D mapping and scene understanding by rendering photo-realistic images. However, existing 3DGS methods often struggle to accurately reconstruct a 3D map that reflects the actual scale and geometry of the real world, which degrades localization performance. To address these limitations, we propose a novel 3DGS method called Geometry-Aware Gaussian Splatting (GeomGS). This method fully integrates LiDAR data into 3D Gaussian primitives via a probabilistic approach, as opposed to approaches that only use LiDAR as initial points or introduce simple constraints for Gaussian points. To this end, we introduce a Geometric Confidence Score (GCS), which identifies the structural reliability of each Gaussian point. The GCS is optimized simultaneously with Gaussians under probabilistic distance constraints to construct a precise structure. Furthermore, we propose a novel localization method that fully utilizes both the geometric and photometric properties of GeomGS. Our GeomGS demonstrates state-of-the-art geometric and localization performance across several benchmarks, while also improving photometric performance.

Figures (4)

• Figure 1: The qualitative results of GeomGS on the KITTI-360 dataset are as follows: (a) 3DGS created with SfM points, (b) 3DGS created with LiDAR points, and (c) GeomGS. The proposed method allows for observing finer details and can address cases where the structure is largely disrupted.
• Figure 2: Overall system of GeomGS. (a), (b) We start with forward-facing images and poses from a dataset. The accumulated LiDAR points, based on the pose, are used as the initial points. (c) We perform geometrically accurate mapping. The parameters of the Gaussian are defined by mean, quaternion, color, and opacity. Additionally, the Geometrically Consistent Score (GCS) is used to identify points that are more geometrically reliable while remaining close to the given LiDAR points. (d) Our localization module fully utilizes LiDAR-based localization and renderable properties of Gaussians to perform iterative localization processes.
• Figure 3: Qualitative comparison of GeomGS and 3DGS in (a) KITTI-360 & (b) KITTI datasets. Patches represent visually distinct regions, highlighting fine details and geometric variations. Our method performs better in various scenarios by incorporating finer details, improving geometric representation, and enhancing overall image quality. The notation adjacent to 3DGS denotes which specific initial point was utilized in the process.
• Figure 4: Comparison of ICP original_icp, WICP, Image Refinement (iNeRF yen2020inerf), and Ours per Iteration. (a) Initial error : 20.0$^\circ$ / 2.0m, (b) Initial error : 25.0$^\circ$ / 5.0m