Multi-Objective Pareto-Front Optimization for Efficient Adaptive VVC Streaming

TL;DR

We address the challenge of adaptive VVC streaming where bitrate, perceptual quality, and decoding complexity must be balanced. We introduce a measurement-driven multi-objective Pareto-front optimization that constructs monotonic, content-aware bitrate ladders and two strategies, Joint Quality-Time PF (JQT-PF) and Joint Rate-Quality-Time PF (JRQT-PF). Across the Inter-4K UHD dataset, JQT-PF achieves about 11.76% average bitrate savings with negligible decoding-time impact at the same $XPSNR$, while JRQT-PF yields around 6.38% bitrate savings with approximately 6.17% decoding-time reduction; more aggressive configurations reach higher gains. This framework outperforms fixed ladders and existing methods, enabling energy-conscious, codec-aware adaptive streaming for VVC and providing a foundation for online integration with RL/MPC-based adaptation.

Abstract

Adaptive video streaming has facilitated improved video streaming over the past years. A balance among coding performance objectives such as bitrate, video quality, and decoding complexity is required to achieve efficient, content- and codec-dependent, adaptive video streaming. This paper proposes a multi-objective Pareto-front (PF) optimization framework to construct quality-monotonic, content-adaptive bitrate ladders Versatile Video Coding (VVC) streaming that jointly optimize video quality, bitrate, and decoding time, which is used as a practical proxy for decoding energy. Two strategies are introduced: the Joint Rate-Quality-Time Pareto Front (JRQT-PF) and the Joint Quality-Time Pareto Front (JQT-PF), each exploring different tradeoff formulations and objective prioritizations. The ladders are constructed under quality monotonicity constraints during adaptive streaming to ensure a consistent Quality of Experience (QoE). Experiments are conducted on a large-scale UHD dataset (Inter-4K), with quality assessed using PSNR, VMAF, and XPSNR, and complexity measured via decoding time and energy consumption. The JQT-PF method achieves 11.76% average bitrate savings while reducing average decoding time by 0.29% to maintain the same XPSNR, compared to a widely-used fixed ladder. More aggressive configurations yield up to 27.88% bitrate savings at the cost of increased complexity. The JRQT-PF strategy, on the other hand, offers more controlled tradeoffs, achieving 6.38 % bitrate savings and 6.17 % decoding time reduction. This framework outperforms existing methods, including fixed ladders, VMAF- and XPSNR-based dynamic resolution selection, and complexity-aware benchmarks. The results confirm that PF optimization with decoding time constraints enables sustainable, high-quality streaming tailored to network and device capabilities.

Figures (7)

• Figure 1: Quality–Rate–Time–Energy points for VVenC/VVdeC vvenc_refVVdeC_ref across spatial resolutions for a quartile of the Inter-4K dataset inter4k_ref. Decoding time and energy consumption were measured using the Running Average Power Limit (RAPL) interface and the CodeCarbon tool codecarbon_ref on a Dell laptop (Intel Core i7-12800H processor, 32 GB RAM).
• Figure 2: Decoding Time–Energy points for VVdeC VVdeC_ref across spatial resolutions for a quartile of the Inter-4K dataset for Dell laptop and Mac Mini.
• Figure 3: Overview of the proposed methodology for the multi-objective PF optimization. Black boxes and lines indicate typical processes and information flow, while blue denotes the particular focus of the proposed method.
• Figure 5: Comparison of BDRX-$\Delta T_{\text{D}}$ tradeoffs of all compared methods. Ideally, negative values for both dimensions would give the best tradeoffs compared to Fixedladder.
• Figure 6: Resulting ladders of four example video sequences using selected benchmark methods, JRQT-PF for $\alpha_M=0.75$, and JQT-PF for $\alpha_J=3$.