Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning Perceptual Representations for Gaming NR-VQA with Multi-Task FR Signals

Yu-Chih Chen, Michael Wang, Chieh-Dun Wen, Kai-Siang Ma, Avinab Saha, Li-Heng Chen, Alan Bovik

TL;DR

This work tackles no-reference VQA for gaming by introducing MTL-VQA, a multi-task pretraining framework that learns perceptual representations from multiple FR metrics without human labels. A shared encoder is trained with adaptive gradient weighting (MGDA/MinNormSolver) across several FR objectives, then frozen to support efficient NR-VQA via a lightweight SVR on temporally pooled features. The approach demonstrates strong label-efficient transfer across gaming datasets, including promising few-shot MOS calibration with as few as 100 labeled clips and competitive performance under PGC-to-UGC domain shifts. It enables practical cloud-gaming QoE monitoring with low-latency NR predictions and points to future work on HUD-aware masking and more robust temporal/artifact-aware auxiliary tasks to improve robustness.

Abstract

No-reference video quality assessment (NR-VQA) for gaming videos is challenging due to limited human-rated datasets and unique content characteristics including fast motion, stylized graphics, and compression artifacts. We present MTL-VQA, a multi-task learning framework that uses full-reference metrics as supervisory signals to learn perceptually meaningful features without human labels for pretraining. By jointly optimizing multiple full-reference (FR) objectives with adaptive task weighting, our approach learns shared representations that transfer effectively to NR-VQA. Experiments on gaming video datasets show MTL-VQA achieves performance competitive with state-of-the-art NR-VQA methods across both MOS-supervised and label-efficient/self-supervised settings.

Learning Perceptual Representations for Gaming NR-VQA with Multi-Task FR Signals

TL;DR

This work tackles no-reference VQA for gaming by introducing MTL-VQA, a multi-task pretraining framework that learns perceptual representations from multiple FR metrics without human labels. A shared encoder is trained with adaptive gradient weighting (MGDA/MinNormSolver) across several FR objectives, then frozen to support efficient NR-VQA via a lightweight SVR on temporally pooled features. The approach demonstrates strong label-efficient transfer across gaming datasets, including promising few-shot MOS calibration with as few as 100 labeled clips and competitive performance under PGC-to-UGC domain shifts. It enables practical cloud-gaming QoE monitoring with low-latency NR predictions and points to future work on HUD-aware masking and more robust temporal/artifact-aware auxiliary tasks to improve robustness.

Abstract

No-reference video quality assessment (NR-VQA) for gaming videos is challenging due to limited human-rated datasets and unique content characteristics including fast motion, stylized graphics, and compression artifacts. We present MTL-VQA, a multi-task learning framework that uses full-reference metrics as supervisory signals to learn perceptually meaningful features without human labels for pretraining. By jointly optimizing multiple full-reference (FR) objectives with adaptive task weighting, our approach learns shared representations that transfer effectively to NR-VQA. Experiments on gaming video datasets show MTL-VQA achieves performance competitive with state-of-the-art NR-VQA methods across both MOS-supervised and label-efficient/self-supervised settings.
Paper Structure (15 sections, 3 equations, 2 figures, 4 tables)

This paper contains 15 sections, 3 equations, 2 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. FR VQA Models
  4. NR VQA Models for Gaming Videos
  5. METHOD
  6. Problem Setup and Data
  7. Task Formulation and Optimization
  8. Domain-Gap Considerations
  9. Architecture and Implementation
  10. Multi-Task FR Supervision and Rationale
  11. Experiments
  12. Experimental Setup
  13. Main Results
  14. Ablation Studies
  15. Conclusion

Figures (2)

  • Figure 1: MTL-VQA overview. A shared encoder is supervised by multiple FR objectives during training phase, then frozen for NR evaluation with a lightweight SVR head on temporally pooled features. Training/evaluation specifics are detailed in Sec. \ref{['sec:problem_setup']}--\ref{['sec:task_opt']}.
  • Figure 2: Source content (blue ‘x’) distribution in paired feature space with corresponding convex hulls (orange boundaries)