EVAN: Evolutional Video Streaming Adaptation via Neural Representation

TL;DR

This work addresses the limitations of traditional ABR by leveraging NeRV neural video representations to enable flexible, progressive bitrate adaptation. It introduces EVAN, a framework that uses Soft Actor-Critic reinforcement learning to choose NeRV representations while enabling progressive playback to avoid rebuffering. Comprehensive experiments on diverse traces demonstrate substantial improvements over baselines, including up to ~50% reductions in rebuffering and QoE gains up to ~25%. The results highlight the potential of neural representations for finer-grained, resilient video streaming, albeit with high NeRV training costs to enable such adaptation.

Abstract

Adaptive bitrate (ABR) using conventional codecs cannot further modify the bitrate once a decision has been made, exhibiting limited adaptation capability. This may result in either overly conservative or overly aggressive bitrate selection, which could cause either inefficient utilization of the network bandwidth or frequent re-buffering, respectively. Neural representation for video (NeRV), which embeds the video content into neural network weights, allows video reconstruction with incomplete models. Specifically, the recovery of one frame can be achieved without relying on the decoding of adjacent frames. NeRV has the potential to provide high video reconstruction quality and, more importantly, pave the way for developing more flexible ABR strategies for video transmission. In this work, a new framework, named Evolutional Video streaming Adaptation via Neural representation (EVAN), which can adaptively transmit NeRV models based on soft actor-critic (SAC) reinforcement learning, is proposed. EVAN is trained with a more exploitative strategy and utilizes progressive playback to avoid re-buffering. Experiments showed that EVAN can outperform existing ABRs with 50% reduction in re-buffering and achieve nearly 20% .

Figures (13)

• Figure 1: Framework of EVAN. The server stores NeRV weights for different representations of video chunks. NeRV weights are transmitted according to their priority. The client can simultaneously reconstruct the video while downloading the associated NeRV weights. The RL agent decides the next chunk's representation to be transmitted, based on the observed network state
• Figure 2: Re-buffering duration over chunks
• Figure 3: Structure of the soft actor-critic network
• Figure 4: Reconstructed frames on different prune ratios
• Figure : (a)