Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LiteVPNet: A Lightweight Network for Video Encoding Control in Quality-Critical Applications

Vibhoothi Vibhoothi, François Pitié, Anil Kokaram

TL;DR

This work addresses precise perceptual quality control for energy-efficient encoding in quality-critical video workflows by predicting AV1 $QP$ values to meet target $VMAF$ scores. It introduces LiteVPNet, a lightweight neural network that fuses bitstream statistics, VCA-based video complexity, and CLIP-based semantic embeddings (via ClipNet) to predict $QP$ across multiple $VMAF$ targets. Results show mean $VMAF$ MAE around 1.0 and high coverage (over 87% within ΔVMAF ≤ 2 and over 96% within ≤4), outperforming prior methods in both accuracy and robustness. The approach delivers substantial speed-ups over brute-force or heavier models, supporting practical deployment in OSVP and other quality-critical streaming contexts, with future work expanding to UHD/HDR content and larger OSVP datasets.

Abstract

In the last decade, video workflows in the cinema production ecosystem have presented new use cases for video streaming technology. These new workflows, e.g. in On-set Virtual Production, present the challenge of requiring precise quality control and energy efficiency. Existing approaches to transcoding often fall short of these requirements, either due to a lack of quality control or computational overhead. To fill this gap, we present a lightweight neural network (LiteVPNet) for accurately predicting Quantisation Parameters for NVENC AV1 encoders that achieve a specified VMAF score. We use low-complexity features, including bitstream characteristics, video complexity measures, and CLIP-based semantic embeddings. Our results demonstrate that LiteVPNet achieves mean VMAF errors below 1.2 points across a wide range of quality targets. Notably, LiteVPNet achieves VMAF errors within 2 points for over 87% of our test corpus, c.f. approx 61% with state-of-the-art methods. LiteVPNet's performance across various quality regions highlights its applicability for enhancing high-value content transport and streaming for more energy-efficient, high-quality media experiences.

LiteVPNet: A Lightweight Network for Video Encoding Control in Quality-Critical Applications

TL;DR

This work addresses precise perceptual quality control for energy-efficient encoding in quality-critical video workflows by predicting AV1 values to meet target scores. It introduces LiteVPNet, a lightweight neural network that fuses bitstream statistics, VCA-based video complexity, and CLIP-based semantic embeddings (via ClipNet) to predict across multiple targets. Results show mean MAE around 1.0 and high coverage (over 87% within ΔVMAF ≤ 2 and over 96% within ≤4), outperforming prior methods in both accuracy and robustness. The approach delivers substantial speed-ups over brute-force or heavier models, supporting practical deployment in OSVP and other quality-critical streaming contexts, with future work expanding to UHD/HDR content and larger OSVP datasets.

Abstract

In the last decade, video workflows in the cinema production ecosystem have presented new use cases for video streaming technology. These new workflows, e.g. in On-set Virtual Production, present the challenge of requiring precise quality control and energy efficiency. Existing approaches to transcoding often fall short of these requirements, either due to a lack of quality control or computational overhead. To fill this gap, we present a lightweight neural network (LiteVPNet) for accurately predicting Quantisation Parameters for NVENC AV1 encoders that achieve a specified VMAF score. We use low-complexity features, including bitstream characteristics, video complexity measures, and CLIP-based semantic embeddings. Our results demonstrate that LiteVPNet achieves mean VMAF errors below 1.2 points across a wide range of quality targets. Notably, LiteVPNet achieves VMAF errors within 2 points for over 87% of our test corpus, c.f. approx 61% with state-of-the-art methods. LiteVPNet's performance across various quality regions highlights its applicability for enhancing high-value content transport and streaming for more energy-efficient, high-quality media experiences.
Paper Structure (10 sections, 3 figures, 2 tables)

This paper contains 10 sections, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Low-complexity feature descriptors for videos
  4. Dataset
  5. LiteVPNet Network Architecture
  6. Results
  7. Ablation Study of the LiteVPNet Input Features
  8. Comparison with other methods
  9. Computational Complexity
  10. Conclusion

Figures (3)

  • Figure 1: Network Design for (a) LiteVPNet Network, (b) ClipNet feature embedding network.
  • Figure 2: Boxplots (25% and 75% percentile) of QP and VMAF prediction errors (y-axis). Each target $QP_i$ (x-axis) corresponds to a VMAF level from 99 down to 80. The average VMAF MAE is 1.0, demonstrating good quality control despite an average QP MAE of 4.5.
  • Figure 3: CDF of VMAF errors for different target VMAF levels (99$\cdots$80). The x-axis indicates the absolute VMAF error, and the y-axis represents the percentage of videos within that error threshold. For example, the QP2 target (VMAF 97) achieves 97% dataset coverage with a VMAF error of $\le3$.