Optimizing Mobile-Friendly Viewport Prediction for Live 360-Degree Video Streaming

TL;DR

This work tackles viewport prediction for live 360-degree video with a mobile-friendly approach. It introduces MFVP, a three-component pipeline that decouples saliency prediction (server-side SalGCN with MAML-based few-shot training) from client-side viewport prediction (a lightweight, SE-augmented ConvLSTM), achieving real-time performance on mobile devices. By integrating MFVP with a class-based bitrate adaptation strategy, the authors demonstrate significant QoE improvements and reduced quality churn in trace-driven experiments. The approach balances prediction accuracy, transmission costs, and mobile computation, enabling scalable live streaming of immersive content. Overall, MFVP advances practical deployment of intelligent viewport prediction for mobile 360° live streaming and yields meaningful QoE gains in realistic network conditions.

Abstract

Viewport prediction is the crucial task for adaptive 360-degree video streaming, as the bitrate control algorithms usually require the knowledge of the user's viewing portions of the frames. Various methods are studied and adopted for viewport prediction from less accurate statistic tools to highly calibrated deep neural networks. Conventionally, it is difficult to implement sophisticated deep learning methods on mobile devices, which have limited computation capability. In this work, we propose an advanced learning-based viewport prediction approach and carefully design it to introduce minimal transmission and computation overhead for mobile terminals. We also propose a model-agnostic meta-learning (MAML) based saliency prediction network trainer, which provides a few-sample fast training solution to obtain the prediction model by utilizing the information from the past models. We further discuss how to integrate this mobile-friendly viewport prediction (MFVP) approach into a typical 360-degree video live streaming system by formulating and solving the bitrate adaptation problem. Extensive experiment results show that our prediction approach can work in real-time for live video streaming and can achieve higher accuracies compared to other existing prediction methods on mobile end, which, together with our bitrate adaptation algorithm, significantly improves the streaming QoE from various aspects. We observe the accuracy of MFVP is 8.1$\%$ to 28.7$\%$ higher than other algorithms and achieves 3.73$\%$ to 14.96$\%$ higher average quality level and 49.6$\%$ to 74.97$\%$ less quality level change than other algorithms.

Figures (18)

• Figure 1: Overview of MFVP
• Figure 2: The Workflow of MAML-based Fast Trainer
• Figure 3: Structure of ConvLSTM model
• Figure 4: Structure of Modified ConvLSTMCell
• Figure 5: Inferring time vs. Sampling frequencies