Analytic Score Optimization for Multi Dimension Video Quality Assessment

TL;DR

A large-scale multi-dimensional VQA dataset UltraVQA is presented that encompasses diverse User-Generated Content~(UGC) annotated across five key quality dimensions: Motion Quality, Motion Amplitude, Aesthetic Quality, Content Quality, and Clarity Quality and introduces Analytic Score Optimization (ASO), a theoretically grounded post-training objective derived for multi-dimensional VQA.

Abstract

Video Quality Assessment (VQA) is evolving beyond single-number mean opinion score toward richer, multi-faceted evaluations of video content. In this paper, we present a large-scale multi-dimensional VQA dataset UltraVQA that encompasses diverse User-Generated Content~(UGC) annotated across five key quality dimensions: Motion Quality, Motion Amplitude, Aesthetic Quality, Content Quality, and Clarity Quality. Each video in our dataset is scored by over 3 human raters on these dimensions, with fine-grained sub-attribute labels, and accompanied by an explanatory rationale generated by GPT based on the collective human judgments. To better leverage these rich annotations and improve discrete quality score assessment, we introduce Analytic Score Optimization (ASO), a theoretically grounded post-training objective derived for multi-dimensional VQA. By reframing quality assessment as a regularized decision-making process, we obtain a closed-form solution that naturally captures the ordinal nature of human ratings, ensuring alignment with human ranking preferences. In experiments, our method outperforms most baselines including closed-source APIs and open-source models, while also reducing mean absolute error (MAE) in quality prediction. Our work highlights the importance of multi-dimensional, interpretable annotations and reinforcement-based alignment in advancing video quality assessment.

Figures (7)

• Figure 1: Cases for each category and distribution of statistics of our UltraVQA.
• Figure 2: Overview of our method. We curate data and human annotators to annotate video quality scores and quality tags with GPT generated reasons. Then we use 2-stage training to elicit the model’s thinking ability before scoring.
• Figure 3: Prompt used for rationale expansion on Motion Quality and Motion Amplitude.
• Figure 4: Prompt used for rationale expansion on Aesthetic Quality.
• Figure 5: Prompt used for rationale expansion on Content Quality.