Subjective and Objective Quality-of-Experience Evaluation Study for Live Video Streaming

TL;DR

An end-to-end QoE evaluation model is proposed, Tao-QoE, which integrates multi-scale semantic features and optical flow-based motion features to predicting a retrospective QoE score, eliminating reliance on statistical quality of service (QoS) features.

Abstract

In recent years, live video streaming has gained widespread popularity across various social media platforms. Quality of experience (QoE), which reflects end-users' satisfaction and overall experience, plays a critical role for media service providers to optimize large-scale live compression and transmission strategies to achieve perceptually optimal rate-distortion trade-off. Although many QoE metrics for video-on-demand (VoD) have been proposed, there remain significant challenges in developing QoE metrics for live video streaming. To bridge this gap, we conduct a comprehensive study of subjective and objective QoE evaluations for live video streaming. For the subjective QoE study, we introduce the first live video streaming QoE dataset, TaoLive QoE, which consists of $42$ source videos collected from real live broadcasts and $1,155$ corresponding distorted ones degraded due to a variety of streaming distortions, including conventional streaming distortions such as compression, stalling, as well as live streaming-specific distortions like frame skipping, variable frame rate, etc. Subsequently, a human study was conducted to derive subjective QoE scores of videos in the TaoLive QoE dataset. For the objective QoE study, we benchmark existing QoE models on the TaoLive QoE dataset as well as publicly available QoE datasets for VoD scenarios, highlighting that current models struggle to accurately assess video QoE, particularly for live content. Hence, we propose an end-to-end QoE evaluation model, Tao-QoE, which integrates multi-scale semantic features and optical flow-based motion features to predicting a retrospective QoE score, eliminating reliance on statistical quality of service (QoS) features.

Figures (6)

• Figure 1: Distortion in actual broadcast scenarios
• Figure 2: Stalling event, accelerated play and frame skipping in TaoLive QoE Database.
• Figure 3: Sample frames of the videos in the proposed TaoLive QoE Database.
• Figure 4: Illustration of the proposed TaoLive QoE database’s MOS distributions from different perspectives
• Figure 5: The overall structure of the proposed network. 1)semantic feature extraction sub-network to extract semantic features from individual input frames; 2)flow motion feature extraction sub-network to extract flow motion features information between frames; 3)multi-scale feature fusion sub-network to process the extracted quality features; 4)feature regression sub-network to predict retrospective QoE score.