Radiant: Large-scale 3D Gaussian Rendering based on Hierarchical Framework

TL;DR

Radiant tackles scalable, privacy-preserving 3D scene reconstruction using 3D Gaussian Splatting in a heterogeneous cloud-edge-device setting. It introduces a three-layer hierarchical framework with Adaptive Region Planning (ARP), Resource-aware Task Partitioning (RTP), and a boundary-aware model aggregation that preserves privacy while maintaining high reconstruction quality. Empirical results on large real-world datasets show up to 25.7% quality improvement and up to 79.6% end-to-end latency reduction over strong baselines. This work enables efficient, scalable, and privacy-conscious large-scale 3D reconstruction suitable for real-world deployments.

Abstract

With the advancement of computer vision, the recently emerged 3D Gaussian Splatting (3DGS) has increasingly become a popular scene reconstruction algorithm due to its outstanding performance. Distributed 3DGS can efficiently utilize edge devices to directly train on the collected images, thereby offloading computational demands and enhancing efficiency. However, traditional distributed frameworks often overlook computational and communication challenges in real-world environments, hindering large-scale deployment and potentially posing privacy risks. In this paper, we propose Radiant, a hierarchical 3DGS algorithm designed for large-scale scene reconstruction that considers system heterogeneity, enhancing the model performance and training efficiency. Via extensive empirical study, we find that it is crucial to partition the regions for each edge appropriately and allocate varying camera positions to each device for image collection and training. The core of Radiant is partitioning regions based on heterogeneous environment information and allocating workloads to each device accordingly. Furthermore, we provide a 3DGS model aggregation algorithm that enhances the quality and ensures the continuity of models' boundaries. Finally, we develop a testbed, and experiments demonstrate that Radiant improved reconstruction quality by up to 25.7\% and reduced up to 79.6\% end-to-end latency.

Figures (13)

• Figure 1: Left: Cloud-based 3DGS training. Middle: Cloud-device-based 3DGS training. Right: Our hierarchical framework for 3DGS training.
• Figure 2: Heterogeneity when training across different scene scales on various devices.
• Figure 3: The overall latency breakdown of each device in two partitioning strategies.
• Figure 4: The performance ($\uparrow$ PSNR, $\uparrow$ SSIM, $\downarrow$ LPIPS) of using model merge and model retrain algorithms in the same area.
• Figure 5: The Overview of Radiant. (a) The Adaptive Region Planning algorithm iterates together with the Resource-aware Task Partitioning algorithm. (b) Device training. (c) Device model aggregation. (d) Edge model aggregation.