CrossView-GS: Cross-view Gaussian Splatting For Large-scale Scene Reconstruction

TL;DR

CrossView-GS tackles cross-view, large-scale scene reconstruction with 3D Gaussian Splatting by building multiple branch priors from different view sets, initializing a cross-view model from distant-view data, and applying gradient-aware regularization guided by pseudo-labels from branches. It further fuses complementary information via a unique Gaussian supplementation step, and fine-tunes the resulting model. Empirical results across aerial-ground, pure aerial, and pure ground cross-view datasets show consistent improvements over state-of-the-art methods in novel view synthesis, with notable gains in aerial views and significant efficiency advantages. The approach offers a practical, scalable path for high-fidelity cross-view reconstructions with applications in VR, smart cities, and GIS, while acknowledging limitations with dynamic objects.

Abstract

3D Gaussian Splatting (3DGS) leverages densely distributed Gaussian primitives for high-quality scene representation and reconstruction. While existing 3DGS methods perform well in scenes with minor view variation, large view changes from cross-view data pose optimization challenges for these methods. To address these issues, we propose a novel cross-view Gaussian Splatting method for large-scale scene reconstruction based on multi-branch construction and fusion. Our method independently reconstructs models from different sets of views as multiple independent branches to establish the baselines of Gaussian distribution, providing reliable priors for cross-view reconstruction during initialization and densification. Specifically, a gradient-aware regularization strategy is introduced to mitigate smoothing issues caused by significant view disparities. Additionally, a unique Gaussian supplementation strategy is utilized to incorporate complementary information of multi-branch into the cross-view model. Extensive experiments on benchmark datasets demonstrate that our method achieves superior performance in novel view synthesis compared to state-of-the-art methods.

Figures (12)

• Figure 1: Left: The cross-view data poses challenges for large-scale scene reconstruction due to significant view variation. Middle: Our method CrossView-GS, demonstrates the ability to reconstruct finer structures with more consistent geometry and appearance than Scaffold-GS scaffold across diverse scenes. Right: The quantitative comparisons with additional SOTA methods highlight the superiority of our method in reconstructing scenes using cross-view data.
• Figure 2: An overview of the proposed method CrossView-GS. The method first initializes the multi-branch structure comprising several sub-models while establishing the cross-view model based on distant views. Then, a gradient-sensitive regularization strategy is used to reconstruct the cross-view model. Finally, unique Gaussian primitives are fused to supplement the cross-view model, achieving large-scale scene reconstruction through fine-tuning.
• Figure 3: The first row shows the variation of maximum gradient in different views during densification when using cross-view or single-view for reconstruction. The second and third rows show the results of Scaffold-GS scaffold using cross-view data and single-view data, respectively. The forth row shows the result of our CrossView-GS using cross-view data.
• Figure 4: The rendering results and zoomed-in images obtained by using our CrossView-GS, as well as some SOTA methods like Hier-GS and Scaffold-GS.
• Figure 5: Qualitative ablations. (a) Cross-view reconstruction using the baseline method. (b) Baseline with initialization from distant aerial views. (c) Baseline with gradient-aware regularization. (d) Baseline with unique Gaussian supplementation.