Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Embracing Radiance Field Rendering in 6G: Over-the-Air Training and Inference with 3D Contents

Guanlin Wu, Zhonghao Lyu, Juyong Zhang, Jie Xu

TL;DR

This work surveys embracing radiance-field rendering (NeRF and 3D-GS) in 6G, addressing over-the-air training and inference across a hierarchical device-edge-cloud architecture. It argues for federated/edge-enabled training, producer-consumer rendering architectures (local/edge/co-rendering), model compression/acceleration, and semantic communications to meet strict latency and bandwidth constraints. Key contributions include a structured overview of NeRF vs. 3D-GS tradeoffs, a hierarchical federated learning framework for large-scale scenes, rendering-architecture options with practical optimization, and a semantic-communication–driven inference design validated by a 3D human-face case study. The paper also outlines radiance-field applications in radio mapping, radar imaging, and multi-modal sensing, highlighting the practical impact for environment-aware wireless networks and immersive 6G experiences.

Abstract

The efficient representation, transmission, and reconstruction of three-dimensional (3D) contents are becoming increasingly important for sixth-generation (6G) networks that aim to merge virtual and physical worlds for offering immersive communication experiences. Neural radiance field (NeRF) and 3D Gaussian splatting (3D-GS) have recently emerged as two promising 3D representation techniques based on radiance field rendering, which are able to provide photorealistic rendering results for complex scenes. Therefore, embracing NeRF and 3D-GS in 6G networks is envisioned to be a prominent solution to support emerging 3D applications with enhanced quality of experience. This paper provides a comprehensive overview on the integration of NeRF and 3D-GS in 6G. First, we review the basics of the radiance field rendering techniques, and highlight their applications and implementation challenges over wireless networks. Next, we consider the over-the-air training of NeRF and 3D-GS models over wireless networks by presenting various learning techniques. We particularly focus on the federated learning design over a hierarchical device-edge-cloud architecture, which is suitable for exploiting distributed data and computing resources over 6G networks to train large models representing large-scale scenes. Then, we consider the over-the-air rendering of NeRF and 3D-GS models at wireless network edge. We present three practical rendering architectures, namely local, remote, and co-rendering, respectively, and provide model compression approaches to facilitate the transmission of radiance field models for rendering. We also present rendering acceleration approaches and joint computation and communication designs to enhance the rendering efficiency. In a case study, we propose a new semantic communication enabled 3D content transmission design.

Embracing Radiance Field Rendering in 6G: Over-the-Air Training and Inference with 3D Contents

TL;DR

This work surveys embracing radiance-field rendering (NeRF and 3D-GS) in 6G, addressing over-the-air training and inference across a hierarchical device-edge-cloud architecture. It argues for federated/edge-enabled training, producer-consumer rendering architectures (local/edge/co-rendering), model compression/acceleration, and semantic communications to meet strict latency and bandwidth constraints. Key contributions include a structured overview of NeRF vs. 3D-GS tradeoffs, a hierarchical federated learning framework for large-scale scenes, rendering-architecture options with practical optimization, and a semantic-communication–driven inference design validated by a 3D human-face case study. The paper also outlines radiance-field applications in radio mapping, radar imaging, and multi-modal sensing, highlighting the practical impact for environment-aware wireless networks and immersive 6G experiences.

Abstract

The efficient representation, transmission, and reconstruction of three-dimensional (3D) contents are becoming increasingly important for sixth-generation (6G) networks that aim to merge virtual and physical worlds for offering immersive communication experiences. Neural radiance field (NeRF) and 3D Gaussian splatting (3D-GS) have recently emerged as two promising 3D representation techniques based on radiance field rendering, which are able to provide photorealistic rendering results for complex scenes. Therefore, embracing NeRF and 3D-GS in 6G networks is envisioned to be a prominent solution to support emerging 3D applications with enhanced quality of experience. This paper provides a comprehensive overview on the integration of NeRF and 3D-GS in 6G. First, we review the basics of the radiance field rendering techniques, and highlight their applications and implementation challenges over wireless networks. Next, we consider the over-the-air training of NeRF and 3D-GS models over wireless networks by presenting various learning techniques. We particularly focus on the federated learning design over a hierarchical device-edge-cloud architecture, which is suitable for exploiting distributed data and computing resources over 6G networks to train large models representing large-scale scenes. Then, we consider the over-the-air rendering of NeRF and 3D-GS models at wireless network edge. We present three practical rendering architectures, namely local, remote, and co-rendering, respectively, and provide model compression approaches to facilitate the transmission of radiance field models for rendering. We also present rendering acceleration approaches and joint computation and communication designs to enhance the rendering efficiency. In a case study, we propose a new semantic communication enabled 3D content transmission design.
Paper Structure (42 sections, 3 equations, 7 figures)

This paper contains 42 sections, 3 equations, 7 figures.

Table of Contents

  1. Introduction
  2. Radiance Field Rendering Over Wireless Networks
  3. Basics of Radiance Field Rendering
  4. NeRF
  5. 3D-GS
  6. Applications in Wireless Networks
  7. Wireless immersive video streaming
  8. 3D-based automation and vision tasks
  9. Wireless sensing and environment-aware communications
  10. Technical Challenges
  11. Intensive computation demands in radiance field training and rendering
  12. Communication overheads for transmission of radiance fields
  13. Critical end-to-end latency requirements
  14. Training of Radiance Field Models over Wireless Networks
  15. Centralized versus Distributed Learning
  16. ...and 27 more sections

Figures (7)

  • Figure 1: Basics of NeRF and 3D-GS based on radiance field rendering.
  • Figure 2: Typical wireless applications of radiance field rendering.
  • Figure 3: Hierarchical federated radiance field learning over 6G networks with end devices, edge servers, and cloud center.
  • Figure 4: Deployment and rendering of radiance fields over 6G networks.
  • Figure 5: Illustration of proposed semantic communications framework for 3D human face transmission with NeRF.
  • ...and 2 more figures