Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

BONES: Near-Optimal Neural-Enhanced Video Streaming

Lingdong Wang, Simran Singh, Jacob Chakareski, Mohammad Hajiesmaili, Ramesh K. Sitaraman

TL;DR

BONES tackles high-quality video delivery under fluctuating networks by integrating neural enhancement into streaming via a Lyapunov-optimized, buffer-occupancy controller. It jointly schedules download quality and enhancement method while tracking two buffers, yielding a drift-plus-penalty online policy with a provable additive gap $O(1/V)$ to the offline optimum. The approach achieves meaningful QoE gains over state-of-the-art ABR and NES baselines in both simulations and a prototype, while offering flexible trade-offs between performance and overhead and maintaining low computational complexity. The work delivers a practical, reproducible baseline for NES and highlights robust performance across diverse network conditions, with code made publicly available for further research.

Abstract

Accessing high-quality video content can be challenging due to insufficient and unstable network bandwidth. Recent advances in neural enhancement have shown promising results in improving the quality of degraded videos through deep learning. Neural-Enhanced Streaming (NES) incorporates this new approach into video streaming, allowing users to download low-quality video segments and then enhance them to obtain high-quality content without violating the playback of the video stream. We introduce BONES, an NES control algorithm that jointly manages the network and computational resources to maximize the quality of experience (QoE) of the user. BONES formulates NES as a Lyapunov optimization problem and solves it in an online manner with near-optimal performance, making it the first NES algorithm to provide a theoretical performance guarantee. Comprehensive experimental results indicate that BONES increases QoE by 5\% to 20\% over state-of-the-art algorithms with minimal overhead. Our code is available at https://github.com/UMass-LIDS/bones.

BONES: Near-Optimal Neural-Enhanced Video Streaming

TL;DR

BONES tackles high-quality video delivery under fluctuating networks by integrating neural enhancement into streaming via a Lyapunov-optimized, buffer-occupancy controller. It jointly schedules download quality and enhancement method while tracking two buffers, yielding a drift-plus-penalty online policy with a provable additive gap to the offline optimum. The approach achieves meaningful QoE gains over state-of-the-art ABR and NES baselines in both simulations and a prototype, while offering flexible trade-offs between performance and overhead and maintaining low computational complexity. The work delivers a practical, reproducible baseline for NES and highlights robust performance across diverse network conditions, with code made publicly available for further research.

Abstract

Accessing high-quality video content can be challenging due to insufficient and unstable network bandwidth. Recent advances in neural enhancement have shown promising results in improving the quality of degraded videos through deep learning. Neural-Enhanced Streaming (NES) incorporates this new approach into video streaming, allowing users to download low-quality video segments and then enhance them to obtain high-quality content without violating the playback of the video stream. We introduce BONES, an NES control algorithm that jointly manages the network and computational resources to maximize the quality of experience (QoE) of the user. BONES formulates NES as a Lyapunov optimization problem and solves it in an online manner with near-optimal performance, making it the first NES algorithm to provide a theoretical performance guarantee. Comprehensive experimental results indicate that BONES increases QoE by 5\% to 20\% over state-of-the-art algorithms with minimal overhead. Our code is available at https://github.com/UMass-LIDS/bones.
Paper Structure (26 sections, 3 theorems, 14 equations, 10 figures, 7 tables, 1 algorithm)

This paper contains 26 sections, 3 theorems, 14 equations, 10 figures, 7 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Problem Formulation
  3. System Model
  4. Optimization Objective
  5. BONES: Control Algorithm and Theoretical Analysis
  6. Control Algorithm
  7. Decision Plane of BONES
  8. Performance Bound
  9. Experimental Evaluation
  10. Performance Metric
  11. Implementation Details
  12. Enhancement Settings
  13. Comparison Algorithms
  14. Simulation Results
  15. Overall Performance
  16. ...and 11 more sections

Key Result

Theorem 1

Assume $Q^d(0)=0, Q^e(0)=0$, and $0 < V \leq \frac{(Q^d_{\max} - p) p}{u_{\max} + \gamma p}$ , where $u_{\max}$ denotes the maximum utility. Then, the following holds: $Q^d(t_k) \leq V \frac{u_{\max} + \gamma p}{p} + p$, and $Q^d(t_k) \leq Q^d_{\max}$.

Figures (10)

  • Figure 1: System models of the traditional ABR algorithm (left) and BONES (right).
  • Figure 2: The decision plane of BONES. Parameter settings from left to right are as follows. "Basic": $\gamma p = 10, \beta = 1$, $1\times$ computation speed. "Faster Computation": $2\times$ computation speed. "Higher $\gamma$": $\gamma p = 50$. "Lower $\beta$": $\beta = 0.5$.
  • Figure 3: Enhancement performance under NAS-MDSR setting.
  • Figure 4: Performance comparison under two enhancement settings. The error bar represents the average visual quality oscillation. Higher quality, lower oscillation, and lower rebuffering ratio are better. * denotes the ABR method is augmented with a greedy enhancement strategy.
  • Figure 5: Performance comparison on different network trace datasets. BONES consistently deliver high QoE across diverse network conditions, including challenging ones.
  • ...and 5 more figures

Theorems & Definitions (3)

  • Theorem 1
  • Theorem 2
  • Lemma 1