Video Quality Assessment for Resolution Cross-Over in Live Sports

TL;DR

This work addresses the challenge of predicting resolution cross-over in ABR for live streaming by moving beyond traditional ACR-based subjective assessments and standard VQMs. It advocates Pair Comparison with Active Sampling and introduces RCQL, a metric that directly measures cross-over quality loss, alongside the LSCO dataset collected from live sports content. Through observer-screening methods and cross-evaluation against VQMs, the authors demonstrate that higher correlation with subjective scores does not guarantee better cross-over accuracy, and they reveal resolution-dependent performance trends for metrics like EQM and PSNR. The practical impact lies in providing a rigorous, cross-over–focused evaluation framework and a live, domain-specific dataset to improve bitrate ladder design for live-streaming services.

Abstract

In adaptive bitrate streaming, resolution cross-over refers to the point on the convex hull where the encoding resolution should switch to achieve better quality. Accurate cross-over prediction is crucial for streaming providers to optimize resolution at given bandwidths. Most existing works rely on objective Video Quality Metrics (VQM), particularly VMAF, to determine the resolution cross-over. However, these metrics have limitations in accurately predicting resolution cross-overs. Furthermore, widely used VQMs are often trained on subjective datasets collected using the Absolute Category Rating (ACR) methodologies, which we demonstrate introduces significant uncertainty and errors in resolution cross-over predictions. To address these problems, we first investigate different subjective methodologies and demonstrate that Pairwise Comparison (PC) achieves better cross-over accuracy than ACR. We then propose a novel metric, Resolution Cross-over Quality Loss (RCQL), to measure the quality loss caused by resolution cross-over errors. Furthermore, we collected a new subjective dataset (LSCO) focusing on live streaming scenarios and evaluated widely used VQMs, by benchmarking their resolution cross-over accuracy.

Figures (5)

• Figure 1: Illustration of resolution cross-over error introduced by ACR: Solid lines show the ground truth cross-over at 1.4 Mbps between 720p and 540p. Dashed lines represent MOS from ACR simulations, with the cross-over at 900 kbps.
• Figure 2: Pilot study example: Comparison of PC vs ACR
• Figure 3: Illustration of RCQL computation for a VQM between two resolutions, $R1$ and $R2$.
• Figure 4: Demonstration of RCQL and $\Delta Bitrate$ between VMAF predictions and subjective scores for the resolution cross-over between 2160p and 1080p.
• Figure 5: Inter-observer consistency for each participant, including 10 synthetic spammers (id from 133 to 143)