Video QoE Metrics from Encrypted Traffic: Application-agnostic Methodology

TL;DR

The paper addresses the problem of estimating user QoE from encrypted traffic in real-time video communication (IMVCAs/VCAs). It proposes an application-agnostic methodology that derives three objective QoE metrics from encrypted streams, using UDP/IP and RTP header features fed into ML models, and validates on a WhatsApp-based dataset totaling 25,680 seconds. Ground-truth FPS is computed with $fps_i = \frac{1}{T} \sum_{k \in i} \mathbb{1}_{[\text{capture}_k \neq \text{capture}_{k-1}]}$, while BRISQUE and PIQE provide spatial quality labels; Random Forest and XGBoost are explored, with Random Forest selected for performance. Results show FPS MAE of 1.7 and PIQE MAE of 2.1, achieving 85.2% accuracy for FPS within ±2 FPS and 90.2% PIQE-rating accuracy, demonstrating practical applicability for network operators to monitor QoE across encrypted traffic.

Abstract

Instant Messaging-Based Video Call Applications (IMVCAs) and Video Conferencing Applications (VCAs) have become integral to modern communication. Ensuring a high Quality of Experience (QoE) for users in this context is critical for network operators, as network conditions significantly impact user QoE. However, network operators lack access to end-device QoE metrics due to encrypted traffic. Existing solutions estimate QoE metrics from encrypted traffic traversing the network, with the most advanced approaches leveraging machine learning models. Subsequently, the need for ground truth QoE metrics for training and validation poses a challenge, as not all video applications provide these metrics. To address this challenge, we propose an application-agnostic approach for objective QoE estimation from encrypted traffic. Independent of the video application, we obtained key video QoE metrics, enabling broad applicability to various proprietary IMVCAs and VCAs. To validate our solution, we created a diverse dataset from WhatsApp video sessions under various network conditions, comprising 25,680 seconds of traffic data and QoE metrics. Our evaluation shows high performance across the entire dataset, with 85.2% accuracy for FPS predictions within an error margin of two FPS, and 90.2% accuracy for PIQE-based quality rating classification.

Figures (7)

• Figure 1: Cumulative distribution function (CDF) of UDP packet size by payload type for the WhatsApp dataset. PT denotes payload type and reTX denotes retransmission.
• Figure 2: Characterization of the spatial quality through CDFs.
• Figure 3: Relationship between bandwidth and (a) mean packet size, and (b) packet count.
• Figure 4: Density distribution of BRISQUE, PIQE and frame rate corresponding to each bandwidth limit: 250 kBps, 125 kBps, 60 kBps, 30 kBps, and 15 kBps.
• Figure 5: QoE metrics predictions during a video session under unexpected bandwidth reductions. From top to bottom, the graphs represent FPS, BRISQUE, PIQE, and bandwidth usage and cap. ML denotes model predictions, GT denotes ground truth and cap denotes limit.