Towards Generalized Video Quality Assessment: A Weak-to-Strong Learning Paradigm

TL;DR

The paper tackles generalization gaps in video quality assessment (VQA) by proposing a weak-to-strong (W2S) learning paradigm that leverages multiple weak teachers to supervise a substantially stronger student, without relying on human annotations. It introduces a learning-to-rank framework to unify heterogeneous supervision signals from homogeneous and synthetic sources and implements iterative self-teaching to progressively expand generalization, recycling progressively stronger students as new teachers. Empirical results on ten in-domain and out-of-distribution benchmarks show the student matching or surpassing the teachers with substantial gains in OOD scenarios, even when trained on pseudo-labels from unlabeled data. The approach demonstrates a scalable path toward VQA foundation models capable of robust quality assessment across diverse content and distortion types without costly human labeling.

Abstract

Video quality assessment (VQA) seeks to predict the perceptual quality of a video in alignment with human visual perception, serving as a fundamental tool for quantifying quality degradation across video processing workflows. The dominant VQA paradigm relies on supervised training with human-labeled datasets, which, despite substantial progress, still suffers from poor generalization to unseen video content. Moreover, its reliance on human annotations -- which are labor-intensive and costly -- makes it difficult to scale datasets for improving model generalization. In this work, we explore weak-to-strong (W2S) learning as a new paradigm for advancing VQA without reliance on large-scale human-labeled datasets. We first provide empirical evidence that a straightforward W2S strategy allows a strong student model to not only match its weak teacher on in-domain benchmarks but also surpass it on out-of-distribution (OOD) benchmarks, revealing a distinct weak-to-strong effect in VQA. Building on this insight, we propose a novel framework that enhances W2S learning from two aspects: (1) integrating homogeneous and heterogeneous supervision signals from diverse VQA teachers -- including off-the-shelf VQA models and synthetic distortion simulators -- via a learn-to-rank formulation, and (2) iterative W2S training, where each strong student is recycled as the teacher in subsequent cycles, progressively focusing on challenging cases. Extensive experiments show that our method achieves state-of-the-art results across both in-domain and OOD benchmarks, with especially strong gains in OOD scenarios. Our findings highlight W2S learning as a principled route to break annotation barriers and achieve scalable generalization in VQA, with implications extending to broader alignment and evaluation tasks.

Figures (11)

• Figure 1: Significant performance drop of state-of-the-art models on out-of-distribution datasets.
• Figure 2: Overview of our weak-to-strong training pipeline.
• Figure 3: Overall architecture of our strong student model. Following LMM-VQA ge2025lmm, we use a dual-branch visual encoder with an additional motion module for temporal distortion modeling. The model supports both single- and dual-video input strategies with distinct training and inference designs.
• Figure 4: Student model performance under pseudo-labels from five weak models: MinimalisticVQA (VII), MinimalisticVQA (IX), FAST-VQA, DOVER, and Q-Align (left to right).
• Figure 5: Our pairwise quality annotations consist of two types: (1) pseudo-labeling based on ensembling homogeneous teachers, and (2) quality ranking derived from integrating heterogeneous teachers.