qAttCNN - Self Attention Mechanism for Video QoE Prediction in Encrypted Traffic

TL;DR

This work tackles QoE prediction under encrypted traffic where DPI is impractical, proposing qAttCNN which combines a learning-based embedding, masked self-attention, and a CNN head to map packet-size streams to no-reference QoE metrics BRISQUE and FPS. By transforming 1D packet data into a 2D representation and leveraging long-range feature relationships, the model achieves superior accuracy (MAEP of 2.14% for BRISQUE and 7.39% for FPS) on a WhatsApp-based dataset using 10-fold cross-validation. Key contributions include novel Embedding and MHSA modules, an effective training regimen with transfer learning and cyclical learning rates, and a thorough ablation study showing the importance of each component. The approach offers a practical pathway for ISPs to monitor QoE over encrypted traffic, though it demands substantial data and computational resources and is currently validated on a single platform, pointing to avenues for broader validation and architectural exploration.

Abstract

The rapid growth of multimedia consumption, driven by major advances in mobile devices since the mid-2000s, has led to widespread use of video conferencing applications (VCAs) such as Zoom and Google Meet, as well as instant messaging applications (IMAs) like WhatsApp and Telegram, which increasingly support video conferencing as a core feature. Many of these systems rely on the Web Real-Time Communication (WebRTC) protocol, enabling direct peer-to-peer media streaming without requiring a third-party server to relay data, reducing the latency and facilitating a real-time communication. Despite WebRTC's potential, adverse network conditions can degrade streaming quality and consequently reduce users' Quality of Experience (QoE). Maintaining high QoE therefore requires continuous monitoring and timely intervention when QoE begins to deteriorate. While content providers can often estimate QoE by directly comparing transmitted and received media, this task is significantly more challenging for internet service providers (ISPs). End-to-end encryption, commonly used by modern VCAs and IMAs, prevent ISPs from accessing the original media stream, leaving only Quality of Service (QoS) and routing information available. To address this limitation, we propose the QoE Attention Convolutional Neural Network (qAttCNN), a model that leverages packet size parameter of the traffic to infer two no-reference QoE metrics viz. BRISQUE and frames per second (FPS). We evaluate qAttCNN on a custom dataset collected from WhatsApp video calls and compare it against existing QoE models. Using mean absolute error percentage (MAEP), our approach achieves 2.14% error for BRISQUE and 7.39% for FPS prediction.

Figures (18)

• Figure 1: Left - Correlation heatmap of the full data packet size features used for prediction. Right - Correlation heatmap of the truncated dataset with 256 first packet sizes (i.e., excluding the non-stationary data tail). The pckt$_\tau$ for $\tau = 1, 2, ... , 350$ represents the size of the packet in time interval $\tau$. As may be noticed, the correlation in the dataset is related to the non-stationary samples.
• Figure 2: Distribution of the BRISQUE labels (left), and the FPS labels (right)
• Figure 3: Two signals --- one stationary (red) and another non-stationary (blue), on a full dataset (left) and on truncated dataset where the last 94 features were removed (right). From the figure we may note that the non-stationary signal drops to 0 value at the last 50 features, while the stationary signal remains high.
• Figure 4: Visualization of the stationary vs nonstationary samples in the dataset with the highest confidence of 1%, as was described in \ref{['tab:df-table']}, where the red lines represent the stationary samples, while the gray nonstationary.
• Figure 5: This figure demonstrates the qAttCNN module an its various components. a) The Embedding module that transforms the 1D input vector $X_{1\times N}$ into $N\times N$ matrix. b) The self attention module, which attends to the most relevant features of the input embedded input vector represented as a 2D matrix (or an image). c) The CNN module that receives the 2D input matrix from the ATT module, and extracts local features from the image by convolution operation and pooling of the strongest features. This module is also responsible for the production of the final prediction