VARFVV: View-Adaptive Real-Time Interactive Free-View Video Streaming with Edge Computing

TL;DR

VARFVV addresses real-time interactive free-view video delivery under constrained bandwidth and edge/computation resources. It introduces an edge-based pipeline that reassembles frames from user-selected view tracks using two representations per view (S for switching and C for constant), avoiding transcoding and enabling low-latency WebRTC streaming. A graph-based popularity predictor (GNN) plus a PPC baseline forecasts view demand, and a Lagrangian-KKT bit-allocation scheme reallocates bits toward perceptually popular views within a budget, maximizing overall QoE. The approach is validated on a dataset of 330 videos across 10 scenes, demonstrating the ability to support over 500 concurrent users with a switching delay as low as 71.5 ms and startup/event-to-eye delays around 195 ms and 0.51 s, respectively. These results indicate VARFVV as a scalable, bandwidth-efficient solution for large-scale mobile UHD FVV deployments, with code and data publicly available.

Abstract

Free-view video (FVV) allows users to explore immersive video content from multiple views. However, delivering FVV poses significant challenges due to the uncertainty in view switching, combined with the substantial bandwidth and computational resources required to transmit and decode multiple video streams, which may result in frequent playback interruptions. Existing approaches, either client-based or cloud-based, struggle to meet high Quality of Experience (QoE) requirements under limited bandwidth and computational resources. To address these issues, we propose VARFVV, a bandwidth- and computationally-efficient system that enables real-time interactive FVV streaming with high QoE and low switching delay. Specifically, VARFVV introduces a low-complexity FVV generation scheme that reassembles multiview video frames at the edge server based on user-selected view tracks, eliminating the need for transcoding and significantly reducing computational overhead. This design makes it well-suited for large-scale, mobile-based UHD FVV experiences. Furthermore, we present a popularity-adaptive bit allocation method, leveraging a graph neural network, that predicts view popularity and dynamically adjusts bit allocation to maximize QoE within bandwidth constraints. We also construct an FVV dataset comprising 330 videos from 10 scenes, including basketball, opera, etc. Extensive experiments show that VARFVV surpasses existing methods in video quality, switching latency, computational efficiency, and bandwidth usage, supporting over 500 users on a single edge server with a switching delay of 71.5ms. Our code and dataset are available at https://github.com/qianghu-huber/VARFVV.

Figures (14)

• Figure 1: Demonstration of our VARFVV. VARFVV is a bandwidth-efficient and low-complexity interactive FVV streaming system and employs an edge server to reassemble multiview frames continuously in time and inter-view based on the user's view track to obtain the FVV stream. VARFVV further delivers the generated FVV stream to the user with low latency through WebRTC. This approach significantly reduces transmission bandwidth and computational resources needed at the edge server and client.
• Figure 2: Overview of the VARFVV System Architecture. Our VARFVV integrates multiview encoding, popularity-adaptive bit allocation, edge-side FVV stream generation, and client-side decoding to deliver high-quality, real-time FVV services with low computational and transmission costs.
• Figure 3: GoP structure design for each stream and view switching strategy.
• Figure 4: Comparison of prediction accuracy between LSTM and GNN methods for view popularity prediction across different numbers of views.
• Figure 5: Graph-based model for view popularity prediction with spatial and temporal attention.