L3GS: Layered 3D Gaussian Splats for Efficient 3D Scene Delivery

TL;DR

This paper tackles efficient delivery of high-quality 3DGaussian Splats (3DGS) scenes over bandwidth-limited networks by introducing Layered 3D Gaussian Splats (L3GS). It develops a training pipeline to produce base and enhancement layers and optionally segment scenes into semantic objects, enabling progressive, view-dependent delivery. A scheduling framework combines a viewport- and bandwidth-aware utility with NP-hard optimization variants, plus tractable single-slot approximations, to decide which splats to download. Complemented by a VR-user viewport predictor and a bandwidth predictor, L3GS demonstrates higher average SSIM (e.g., $16.9\%$ over baselines) with low latency across eight scenes and realistic traces, and proves compatibility with alternative 3DGS representations. The work enables scalable, high-fidelity 3D scene delivery suitable for real-time VR/AR use and provides a flexible platform for integrating future 3DGS formats.

Abstract

Traditional 3D content representations include dense point clouds that consume large amounts of data and hence network bandwidth, while newer representations such as neural radiance fields suffer from poor frame rates due to their non-standard volumetric rendering pipeline. 3D Gaussian splats (3DGS) can be seen as a generalization of point clouds that meet the best of both worlds, with high visual quality and efficient rendering for real-time frame rates. However, delivering 3DGS scenes from a hosting server to client devices is still challenging due to high network data consumption (e.g., 1.5 GB for a single scene). The goal of this work is to create an efficient 3D content delivery framework that allows users to view high quality 3D scenes with 3DGS as the underlying data representation. The main contributions of the paper are: (1) Creating new layered 3DGS scenes for efficient delivery, (2) Scheduling algorithms to choose what splats to download at what time, and (3) Trace-driven experiments from users wearing virtual reality headsets to evaluate the visual quality and latency. Our system for Layered 3D Gaussian Splats delivery L3GS demonstrates high visual quality, achieving 16.9% higher average SSIM compared to baselines, and also works with other compressed 3DGS representations.

Figures (20)

• Figure 1: System architecture. Given a set of layered and segmented 3D Gaussian splats, L3GS retrieves the most useful splats within the user's predicted viewport and network bandwidth.
• Figure 2: Our approaches ("Ours" and "Separate") can gracefully trade off visual quality for the number of splats. Data from the "Train" scene knapitsch2017tanksntemple.
• Figure 3: Training framework for layered 3DGS. The layers are iteratively trained, with subsequent layers relying on frozen splats from preceding layers.
• Figure 4: Illustration of the utility function (\ref{['eqn:utility']}) for two objects (egg and pork belly) from the "Ramen" scene ye2024gaussiangroupingsegmentedit.
• Figure 5: Real viewport trajectory data from the bicycle scene, each color representing a different user.

Theorems & Definitions (4)

• Lemma 1
• Theorem 1
• Lemma 2
• Theorem 2