From Capture to Display: A Survey on Volumetric Video

TL;DR

This survey provides a comprehensive overview of volumetric video from capture to display, outlining representations, datasets, and quality assessment, and detailing the end-to-end pipeline (capturing, compression, transmission, rendering, display). It categorizes three main pipeline streams—tile-based, layered, and super-resolution transmission—alongside rendering strategies (point cloud, mesh, NeRF) and discusses local versus remote rendering, privacy concerns, and edge computing integration. Key contributions include organizing the literature into a cohesive framework, summarizing current open datasets and evaluation methodologies, and highlighting critical challenges such as real-time NeRF rendering, viewport-aware streaming, and unified benchmarks. The findings emphasize that advances in representations, compression efficiency, and streaming optimization are essential to realize practical, scalable volumetric-video services across mobile and edge devices with broad applications in telepresence, rehabilitation, and education.

Abstract

Volumetric video, which offers immersive viewing experiences, is gaining increasing prominence. With its six degrees of freedom, it provides viewers with greater immersion and interactivity compared to traditional videos. Despite their potential, volumetric video services pose significant challenges. This survey conducts a comprehensive review of the existing literature on volumetric video. We firstly provide a general framework of volumetric video services, followed by a discussion on prerequisites for volumetric video, encompassing representations, open datasets, and quality assessment metrics. Then we delve into the current methodologies for each stage of the volumetric video service pipeline, detailing capturing, compression, transmission, rendering, and display techniques. Lastly, we explore various applications enabled by this pioneering technology and we present an array of research challenges and opportunities in the domain of volumetric video services. This survey aspires to provide a holistic understanding of this burgeoning field and shed light on potential future research trajectories, aiming to bring the vision of volumetric video to fruition.

Figures (6)

• Figure 1: Overview of volumetric video delivery systems.
• Figure 2: Organization of this survey.
• Figure 3: A general framework for volumetric video service.
• Figure 4: End-to-end pipeline of volumetric video services.
• Figure 5: Illustrations for camera array setup.