Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Edge Collaborative Gaussian Splatting with Integrated Rendering and Communication

Yujie Wan, Chenxuan Liu, Shuai Wang, Tong Zhang, James Jianqiao Yu, Kejiang Ye, Dusit Niyato, Chengzhong Xu

TL;DR

The paper addresses rendering quality challenges of Gaussian splatting (GS) on resource-limited devices by introducing edge collaborative GS (ECO-GS), where users can switch between local small GS and remote edge GS. It formulates integrated rendering and communication (IRAC) to jointly optimize binary collaboration indicators $\mathbf{x}$ and edge transmit powers $\mathbf{p}$ under latency and bandwidth constraints, using a GS switching function as a tractable surrogate for rendering fidelity. Two algorithms are proposed: Penalty Majorization Minimization (PMM) to find high-quality solutions, and Imitation Learning Optimization (ILO) to achieve real-time decisions by mimicking PMM with a lightweight neural network. Experiments demonstrate IRAC outperforms baselines in image fidelity while respecting latency, with ILO delivering real-time performance (≈$0.1$ ms inference) and ~$100\times$ speedups over PMM, highlighting the practical impact of selective edge collaboration for scalable edge rendering.

Abstract

Gaussian splatting (GS) struggles with degraded rendering quality on low-cost devices. To address this issue, we present edge collaborative GS (ECO-GS), where each user can switch between a local small GS model to guarantee timeliness and a remote large GS model to guarantee fidelity. However, deciding how to engage the large GS model is nontrivial, due to the interdependency between rendering requirements and resource conditions. To this end, we propose integrated rendering and communication (IRAC), which jointly optimizes collaboration status (i.e., deciding whether to engage large GS) and edge power allocation (i.e., enabling remote rendering) under communication constraints across different users by minimizing a newly-derived GS switching function. Despite the nonconvexity of the problem, we propose an efficient penalty majorization minimization (PMM) algorithm to obtain the critical point solution. Furthermore, we develop an imitation learning optimization (ILO) algorithm, which reduces the computational time by over 100x compared to PMM. Experiments demonstrate the superiority of PMM and the real-time execution capability of ILO.

Edge Collaborative Gaussian Splatting with Integrated Rendering and Communication

TL;DR

The paper addresses rendering quality challenges of Gaussian splatting (GS) on resource-limited devices by introducing edge collaborative GS (ECO-GS), where users can switch between local small GS and remote edge GS. It formulates integrated rendering and communication (IRAC) to jointly optimize binary collaboration indicators and edge transmit powers under latency and bandwidth constraints, using a GS switching function as a tractable surrogate for rendering fidelity. Two algorithms are proposed: Penalty Majorization Minimization (PMM) to find high-quality solutions, and Imitation Learning Optimization (ILO) to achieve real-time decisions by mimicking PMM with a lightweight neural network. Experiments demonstrate IRAC outperforms baselines in image fidelity while respecting latency, with ILO delivering real-time performance (≈ ms inference) and ~ speedups over PMM, highlighting the practical impact of selective edge collaboration for scalable edge rendering.

Abstract

Gaussian splatting (GS) struggles with degraded rendering quality on low-cost devices. To address this issue, we present edge collaborative GS (ECO-GS), where each user can switch between a local small GS model to guarantee timeliness and a remote large GS model to guarantee fidelity. However, deciding how to engage the large GS model is nontrivial, due to the interdependency between rendering requirements and resource conditions. To this end, we propose integrated rendering and communication (IRAC), which jointly optimizes collaboration status (i.e., deciding whether to engage large GS) and edge power allocation (i.e., enabling remote rendering) under communication constraints across different users by minimizing a newly-derived GS switching function. Despite the nonconvexity of the problem, we propose an efficient penalty majorization minimization (PMM) algorithm to obtain the critical point solution. Furthermore, we develop an imitation learning optimization (ILO) algorithm, which reduces the computational time by over 100x compared to PMM. Experiments demonstrate the superiority of PMM and the real-time execution capability of ILO.
Paper Structure (6 sections, 1 theorem, 12 equations, 7 figures, 1 table)

This paper contains 6 sections, 1 theorem, 12 equations, 7 figures, 1 table.

Table of Contents

  1. Introduction
  2. Problem Formulation
  3. Penalty Majorizaiton Minimization
  4. Imitation Learning Optimization
  5. Experiments
  6. Conclusion

Key Result

Proposition 1

The function $\widehat{\varphi}$ satisfies the following: (i) Convexity: $\widehat{\varphi}(\mathbf x|\mathbf x^{[n]} )$ is convex in $\mathbf{x}$; (ii) Upper bound: $\widehat{\varphi}(\mathbf x|\mathbf x^{[n]} )\geq \varphi(\mathbf{x})$; (iii) Local equivalence:

Figures (7)

  • Figure 1: The ECO-GS system.
  • Figure 2: Pipeline of the ILO-IRAC algorithm.
  • Figure 3: Quantitative comparison of different schemes.
  • Figure 4: Comparison of rendered images.
  • Figure 5: Case study for PMM and Greedy schemes.
  • ...and 2 more figures

Theorems & Definitions (1)

  • Proposition 1