GauU-Scene: A Scene Reconstruction Benchmark on Large Scale 3D Reconstruction Dataset Using Gaussian Splatting

TL;DR

This work targets large-scale urban 3D reconstruction by introducing GauU-Scene, a Gaussian Splatting benchmark built on the U-Scene dataset. It leverages drone-captured RGB data and high-precision LiDAR to enable rooftop-inclusive city-scale reconstructions over more than 1.5 square kilometers, and proposes a LiDAR-Image Fusion approach to provide priors for Gaussian Splatting. The study presents quantitative benchmarks (PSNR, L1) and qualitative results showing improvements when LiDAR priors are fused with imagery, highlighting the value of multi-modal data for accurate 3D modeling. The dataset and benchmark advance evaluation of Gaussian Splatting in drone-derived, large-scale environments and identify edge artifacts and scalability as key areas for future work.

Abstract

We introduce a novel large-scale scene reconstruction benchmark using the newly developed 3D representation approach, Gaussian Splatting, on our expansive U-Scene dataset. U-Scene encompasses over one and a half square kilometres, featuring a comprehensive RGB dataset coupled with LiDAR ground truth. For data acquisition, we employed the Matrix 300 drone equipped with the high-accuracy Zenmuse L1 LiDAR, enabling precise rooftop data collection. This dataset, offers a unique blend of urban and academic environments for advanced spatial analysis convers more than 1.5 km$^2$. Our evaluation of U-Scene with Gaussian Splatting includes a detailed analysis across various novel viewpoints. We also juxtapose these results with those derived from our accurate point cloud dataset, highlighting significant differences that underscore the importance of combine multi-modal information

Figures (5)

• Figure 1: Our dataset is divided into three main parts. The first part is the top part of this graph. We call it CUHKSZ(The Chinese University of Hong Kong, Shenzhen) lower campus, and the bottom left corner shows the upper campus of CUHKSZ, and the bottom right corner shows the SMBU(Shenzhen MSU-BIT University) Campus. We use highly accurate lidar to collect the dataset and the range we cover is more than 1.5 $km^2$.
• Figure 2: The current point cloud registration method usually cannot handle different scales, so we first scale the raw point cloud to the same size as the SfM sparse point cloud. To do this, we find the maximum distance or variance in the SfM, as there are always some points far from the center in SfM. Then, we perform coarse matching manually and fine-tune it using ICP (Iterative Closest Point).
• Figure 3: The left one is the quality of the point if the point is blue, then the quality is ok, otherwise it is red. The right-hand side picture shows the point's altitude, and the middle one is the RGB point cloud
• Figure 4: The three figures here give another angle for our raw point cloud dataset
• Figure 5: The left hand-side of this picture shows the result of using vanilla Gaussian Splatting, while the right hand side shows the result of using lidar-fused Gaussian Splatting. One can easily find the left hand-side pictures contains a some blurry black cloud, and also has an irregular blur building, while the right handside does not contain this defect