3DGS-ReLoc: 3D Gaussian Splatting for Map Representation and Visual ReLocalization

Abstract

This paper presents a novel system designed for 3D mapping and visual relocalization using 3D Gaussian Splatting. Our proposed method uses LiDAR and camera data to create accurate and visually plausible representations of the environment. By leveraging LiDAR data to initiate the training of the 3D Gaussian Splatting map, our system constructs maps that are both detailed and geometrically accurate. To mitigate excessive GPU memory usage and facilitate rapid spatial queries, we employ a combination of a 2D voxel map and a KD-tree. This preparation makes our method well-suited for visual localization tasks, enabling efficient identification of correspondences between the query image and the rendered image from the Gaussian Splatting map via normalized cross-correlation (NCC). Additionally, we refine the camera pose of the query image using feature-based matching and the Perspective-n-Point (PnP) technique. The effectiveness, adaptability, and precision of our system are demonstrated through extensive evaluation on the KITTI360 dataset.

Figures (7)

• Figure 1: Pipeline of 3D Gaussian Splatting for Map Representation and Visual ReLocalization: The process starts by creating a colorized point cloud map from LiDAR scans, images, and poses. This map serves as the initialization for the 3D Gaussian Splatting (3DGS) map, which is incrementally trained on submaps. The 3DGS map is stored as a 2D voxel map, with a KD-tree enabling rapid spatial queries. For relocalization, a submap proximate to the query image's coarse pose is selected to render a series of images and depths. The query image is then subjected to a brute-force search against this image sequence to find the closest rendered image and depth. Subsequently, feature-based matching and the Perspective-n-Point (PnP) method are employed to iteratively refine the query image's pose, achieving precise localization within the global map.
• Figure 2: (a)/(e) Illustrating the Relationship between X/Yaw Error and Normalized Cross-Correlation in Localization Initialization; (b)/(f) Query Image for Localization; (c)/(g) Best Matches in Rendered Image Sequences; (d)/(h) Worst Matches in Rendered Image Sequences.
• Figure 3: Evaluation of Initial Localization X, Y, Yaw Error Histogram
• Figure 4: Evaluation of Re-Localization X, Y, Yaw Error Histogram
• Figure 5: Comparison of Ground Truth and Predicted Trajectories from Six Views: Roll, Pitch, Yaw, X, Y, Z for Sequence 0