CoSurfGS:Collaborative 3D Surface Gaussian Splatting with Distributed Learning for Large Scene Reconstruction

TL;DR

CoSurfGS presents a device-edge-cloud framework for large-scale surface reconstruction using 3D Gaussian Splatting. It introduces Local Model Compression and a Model Aggregation Scheme to enable parallel training, reduce memory, and maintain geometry via self-distillation across local models. The method leverages staged device training with single- and multi-view geometric constraints, adaptive pruning, and depth/normal supervision to achieve state-of-the-art surface reconstruction while delivering fast training and privacy preservation. Experiments on BlendedMVS, UrbanScene3D, and Mill-19 demonstrate superior geometry, competitive novel-view synthesis, and substantial efficiency gains over existing centralized and distributed baselines.

Abstract

3D Gaussian Splatting (3DGS) has demonstrated impressive performance in scene reconstruction. However, most existing GS-based surface reconstruction methods focus on 3D objects or limited scenes. Directly applying these methods to large-scale scene reconstruction will pose challenges such as high memory costs, excessive time consumption, and lack of geometric detail, which makes it difficult to implement in practical applications. To address these issues, we propose a multi-agent collaborative fast 3DGS surface reconstruction framework based on distributed learning for large-scale surface reconstruction. Specifically, we develop local model compression (LMC) and model aggregation schemes (MAS) to achieve high-quality surface representation of large scenes while reducing GPU memory consumption. Extensive experiments on Urban3d, MegaNeRF, and BlendedMVS demonstrate that our proposed method can achieve fast and scalable high-fidelity surface reconstruction and photorealistic rendering. Our project page is available at \url{https://gyy456.github.io/CoSurfGS}.

Figures (10)

• Figure 1: Our proposed CoSurfGS serves as a "device-edge-cloud" distributed learning framework that enables multi-agent parallel training. Under this framework, we can achieve superior large-scene reconstruction performance w.r.t the novel view synthesis, depth rendering, and surface normal prediction results (see the bottom part). Meanwhile, this framework can also accelerate the whole modeling process while preserving the privacy of local regions.
• Figure 2: Our CoSurfGS follows "device-edge-cloud" three-layer distributed architecture. On the device side, each device is responsible for reconstructing an individual area by capturing images, performing SfM to initialize both extrinsic and intrinsic, and training the Gaussian models $\mathbf{G}^L$. On the edge side, devices upload their Gaussian model to the edge followed by two-step aggregation techniques: 1) Local Model Compression (LMC) module prunes the redundant Gaussian points and abandons images with few contributions on reconstruction area; 2) Model Aggregation Scheme (MAS) module uses a self-distillation technique to aggregate the compressed model into a global model $\mathbf{G}^G$. Moreover, the edge-cloud shares the same process as we've done on the edge side.
• Figure 3: 3D mesh comparison between our method and other surface reconstruction methods. The result of Scene-01, Scene-02, Scene-03, and Scene-04 are represented from top to bottom. The discriminate area are zoomed up by '$\Box$'.
• Figure 4: Qualitative results of our method and other methods in image and depth rendering, it shows the result of Rubble and Building, other large scenes visualization can be seen in the Supp. \ref{['sec: Additional depth']}.
• Figure 5: Qualitative results of Model Aggregation Scheme in Rubble dataset.