Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Green Robotic Mixed Reality with Gaussian Splatting

Chenxuan Liu, He Li, Zongze Li, Shuai Wang, Wei Xu, Kejiang Ye, Derrick Wing Kwan Ng, Chengzhong Xu

TL;DR

The paper tackles the high energy cost of RoboMR caused by transmitting high-resolution sensor data by introducing Gaussian Splatting RobomR (GSRMR), which leverages a memory of historical images to synthesize a photo-realistic view from the robot pose. It formulates a cross-layer optimization, $\mathsf{P}_{\mathrm{GS}}$, that jointly selects content switching $x_t$ and uplink power $p_t$, with data budget $D_t = x_t I + (1-x_t) S$ and a fidelity constraint $\frac{1}{T} \sum_t L_t (1 - x_t) \le L_{\mathrm{th}}$, and solves it with Accelerated Penalty Optimization (APO) that uses a DC surrogate $\widehat{\varphi}(\mathbf{x}|\mathbf{x}^{[n]})$. APO combines penalty relaxation of binary variables with a DC algorithm and a ranking-based initialization to achieve fast convergence; per-iteration complexity scales as $\mathcal{O}((2T)^{3.5})$. Experiments on ROS and CARLA show energy savings of over 90% compared with RoboMR and PSNR/SSIM gains of at least 6 dB and notable perceptual quality improvements, while APO outperforms several baselines and minimizes GS discrepancies. This work demonstrates a viable path to green RoboMR by integrating memory-based rendering with cross-layer optimization for adaptive content delivery.

Abstract

Realizing green communication in robotic mixed reality (RoboMR) systems presents a challenge, due to the necessity of uploading high-resolution images at high frequencies through wireless channels. This paper proposes Gaussian splatting (GS) RoboMR (GSRMR), which achieves a lower energy consumption and makes a concrete step towards green RoboMR. The crux to GSRMR is to build a GS model which enables the simulator to opportunistically render a photo-realistic view from the robot's pose, thereby reducing the need for excessive image uploads. Since the GS model may involve discrepancies compared to the actual environments, a GS cross-layer optimization (GSCLO) framework is further proposed, which jointly optimizes content switching (i.e., deciding whether to upload image or not) and power allocation across different frames. The GSCLO problem is solved by an accelerated penalty optimization (APO) algorithm. Experiments demonstrate that the proposed GSRMR reduces the communication energy by over 10x compared with RoboMR. Furthermore, the proposed GSRMR with APO outperforms extensive baseline schemes, in terms of peak signal-to-noise ratio (PSNR) and structural similarity index measure (SSIM).

Green Robotic Mixed Reality with Gaussian Splatting

TL;DR

The paper tackles the high energy cost of RoboMR caused by transmitting high-resolution sensor data by introducing Gaussian Splatting RobomR (GSRMR), which leverages a memory of historical images to synthesize a photo-realistic view from the robot pose. It formulates a cross-layer optimization, , that jointly selects content switching and uplink power , with data budget and a fidelity constraint , and solves it with Accelerated Penalty Optimization (APO) that uses a DC surrogate . APO combines penalty relaxation of binary variables with a DC algorithm and a ranking-based initialization to achieve fast convergence; per-iteration complexity scales as . Experiments on ROS and CARLA show energy savings of over 90% compared with RoboMR and PSNR/SSIM gains of at least 6 dB and notable perceptual quality improvements, while APO outperforms several baselines and minimizes GS discrepancies. This work demonstrates a viable path to green RoboMR by integrating memory-based rendering with cross-layer optimization for adaptive content delivery.

Abstract

Realizing green communication in robotic mixed reality (RoboMR) systems presents a challenge, due to the necessity of uploading high-resolution images at high frequencies through wireless channels. This paper proposes Gaussian splatting (GS) RoboMR (GSRMR), which achieves a lower energy consumption and makes a concrete step towards green RoboMR. The crux to GSRMR is to build a GS model which enables the simulator to opportunistically render a photo-realistic view from the robot's pose, thereby reducing the need for excessive image uploads. Since the GS model may involve discrepancies compared to the actual environments, a GS cross-layer optimization (GSCLO) framework is further proposed, which jointly optimizes content switching (i.e., deciding whether to upload image or not) and power allocation across different frames. The GSCLO problem is solved by an accelerated penalty optimization (APO) algorithm. Experiments demonstrate that the proposed GSRMR reduces the communication energy by over 10x compared with RoboMR. Furthermore, the proposed GSRMR with APO outperforms extensive baseline schemes, in terms of peak signal-to-noise ratio (PSNR) and structural similarity index measure (SSIM).

Paper Structure

This paper contains 11 sections, 2 theorems, 19 equations, 5 figures, 1 table.

Table of Contents

  1. Introduction
  2. Problem Formulation
  3. RoboMR System
  4. Green RoboMR System
  5. Gaussian Splatting RoboMR
  6. Accelerated Penalty Optimization
  7. Penalty for Binary Constraints
  8. Difference of Convex Algorithm
  9. Ranking-based Initialization
  10. Experiments
  11. Conclusion

Key Result

Proposition 1

$\widehat{\varphi}(\mathbf x|\mathbf x^\star )$ satisfy the following conditions: (i) Upper bound: $\widehat{\varphi}(\mathbf x|\mathbf x^\star ) \geq \varphi(\mathbf x)$ for any $\mathbf x$. (ii) Convexity: $\widehat{\varphi}(\mathbf x|\mathbf x^\star )$ is convex in $\mathbf{x}$. (iii) Equality: $

Figures (5)

  • Figure 1: System model of RoboMR.
  • Figure 2: Architecture of GS-RoboMR.
  • Figure 3: Implementation of the RoboMR platform.
  • Figure 4: (a) Convergence behavior of APO. (b) Comparison of energy consumption.
  • Figure 5: (a) Left-hand side: The MR scenario, where the blue line represents robot trajectory, red boxes represent start and end positions, green boxes represent image poses, and blue boxes represent positions of virtual agents, respectively. Right-hand side: Qualitative comparison of 6 image frames. (b) Content switching $\{x_t\}$ and power profiles $\{p_t\}$.

Theorems & Definitions (4)

  • Proposition 1
  • proof
  • Proposition 2
  • proof