Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

GaussReg: Fast 3D Registration with Gaussian Splatting

Jiahao Chang, Yinglin Xu, Yihao Li, Yuantao Chen, Xiaoguang Han

TL;DR

GaussReg addresses the challenging problem of registering 3D Gaussian Splatting (GS) scene models by introducing a fast, coarse-to-fine pipeline that first aligns GS-derived point clouds and then refines with image-guided volumetric features drawn from GS-rendered overlap views, culminating in a fused GS model. The method leverages a GS-aware coarse registration, an overlap-driven image selection, and an image-guided 3D feature extraction process to achieve high geometric accuracy with substantial speed gains over baselines such as HLoc. A new ScanNet-GSReg benchmark and a wild GSReg dataset enable comprehensive evaluation across indoor and outdoor scenes, with GaussReg attaining state-of-the-art performance on multiple datasets and demonstrating a 44× speedup relative to HLoc. The work advances GS-based scene representations for large-scale 3D reconstruction by providing an effective registration framework, robust generalization to objects, and a practical fusion mechanism for combining GS models. The proposed approach has practical implications for scalable NeRF-based scene stitching and continual scene modification in real-world applications.

Abstract

Point cloud registration is a fundamental problem for large-scale 3D scene scanning and reconstruction. With the help of deep learning, registration methods have evolved significantly, reaching a nearly-mature stage. As the introduction of Neural Radiance Fields (NeRF), it has become the most popular 3D scene representation as its powerful view synthesis capabilities. Regarding NeRF representation, its registration is also required for large-scale scene reconstruction. However, this topic extremly lacks exploration. This is due to the inherent challenge to model the geometric relationship among two scenes with implicit representations. The existing methods usually convert the implicit representation to explicit representation for further registration. Most recently, Gaussian Splatting (GS) is introduced, employing explicit 3D Gaussian. This method significantly enhances rendering speed while maintaining high rendering quality. Given two scenes with explicit GS representations, in this work, we explore the 3D registration task between them. To this end, we propose GaussReg, a novel coarse-to-fine framework, both fast and accurate. The coarse stage follows existing point cloud registration methods and estimates a rough alignment for point clouds from GS. We further newly present an image-guided fine registration approach, which renders images from GS to provide more detailed geometric information for precise alignment. To support comprehensive evaluation, we carefully build a scene-level dataset called ScanNet-GSReg with 1379 scenes obtained from the ScanNet dataset and collect an in-the-wild dataset called GSReg. Experimental results demonstrate our method achieves state-of-the-art performance on multiple datasets. Our GaussReg is 44 times faster than HLoc (SuperPoint as the feature extractor and SuperGlue as the matcher) with comparable accuracy.

GaussReg: Fast 3D Registration with Gaussian Splatting

TL;DR

GaussReg addresses the challenging problem of registering 3D Gaussian Splatting (GS) scene models by introducing a fast, coarse-to-fine pipeline that first aligns GS-derived point clouds and then refines with image-guided volumetric features drawn from GS-rendered overlap views, culminating in a fused GS model. The method leverages a GS-aware coarse registration, an overlap-driven image selection, and an image-guided 3D feature extraction process to achieve high geometric accuracy with substantial speed gains over baselines such as HLoc. A new ScanNet-GSReg benchmark and a wild GSReg dataset enable comprehensive evaluation across indoor and outdoor scenes, with GaussReg attaining state-of-the-art performance on multiple datasets and demonstrating a 44× speedup relative to HLoc. The work advances GS-based scene representations for large-scale 3D reconstruction by providing an effective registration framework, robust generalization to objects, and a practical fusion mechanism for combining GS models. The proposed approach has practical implications for scalable NeRF-based scene stitching and continual scene modification in real-world applications.

Abstract

Point cloud registration is a fundamental problem for large-scale 3D scene scanning and reconstruction. With the help of deep learning, registration methods have evolved significantly, reaching a nearly-mature stage. As the introduction of Neural Radiance Fields (NeRF), it has become the most popular 3D scene representation as its powerful view synthesis capabilities. Regarding NeRF representation, its registration is also required for large-scale scene reconstruction. However, this topic extremly lacks exploration. This is due to the inherent challenge to model the geometric relationship among two scenes with implicit representations. The existing methods usually convert the implicit representation to explicit representation for further registration. Most recently, Gaussian Splatting (GS) is introduced, employing explicit 3D Gaussian. This method significantly enhances rendering speed while maintaining high rendering quality. Given two scenes with explicit GS representations, in this work, we explore the 3D registration task between them. To this end, we propose GaussReg, a novel coarse-to-fine framework, both fast and accurate. The coarse stage follows existing point cloud registration methods and estimates a rough alignment for point clouds from GS. We further newly present an image-guided fine registration approach, which renders images from GS to provide more detailed geometric information for precise alignment. To support comprehensive evaluation, we carefully build a scene-level dataset called ScanNet-GSReg with 1379 scenes obtained from the ScanNet dataset and collect an in-the-wild dataset called GSReg. Experimental results demonstrate our method achieves state-of-the-art performance on multiple datasets. Our GaussReg is 44 times faster than HLoc (SuperPoint as the feature extractor and SuperGlue as the matcher) with comparable accuracy.
Paper Structure (30 sections, 7 equations, 5 figures, 6 tables)

This paper contains 30 sections, 7 equations, 5 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. 3D point cloud registration
  4. 3D scene representation
  5. NeRF Registration
  6. Method
  7. Overview
  8. Coarse Registration
  9. Training Strategy and Loss Function
  10. Image-Guided Fine Registration
  11. Overlap Image Selection
  12. Image-Guided 3D Feature Extraction
  13. Training Strategy and Loss Function
  14. Gaussian Splatting Fusion and Filtering
  15. Experiment
  16. ...and 15 more sections

Figures (5)

  • Figure 1: The purpose of our method is to register scenes A and B with Gaussian Splattingkerbl3Dgaussians models, and then combine A with B to get the fused Gaussian Splatting model. The first row is the visualization of the 3D Gaussians.
  • Figure 2: The architecture of GaussReg. Please refer to the text for detailed architecture.
  • Figure 3: The illustration of our overlap image selection and I3D feature extraction.
  • Figure 4: Visualization of our final registration results on ScanNet-GSReg and GSReg. The first two columns are visualizations of GS point clouds to be registered. The last two columns are visualizations of our final registration and ground-truth results.
  • Figure 5: Quantitative Results on the GSReg dataset. The first two rows are indoor scenes, and the last two rows are outdoor scenes. The first and third columns are rendering images from GS kerbl3Dgaussians models of Scene A and Scene B. The second and last columns are rendering images from our fused GS model.