NTIRE 2024 Challenge on Short-form UGC Video Quality Assessment: Methods and Results
Xin Li, Kun Yuan, Yajing Pei, Yiting Lu, Ming Sun, Chao Zhou, Zhibo Chen, Radu Timofte, Wei Sun, Haoning Wu, Zicheng Zhang, Jun Jia, Zhichao Zhang, Linhan Cao, Qiubo Chen, Xiongkuo Min, Weisi Lin, Guangtao Zhai, Jianhui Sun, Tianyi Wang, Lei Li, Han Kong, Wenxuan Wang, Bing Li, Cheng Luo, Haiqiang Wang, Xiangguang Chen, Wenhui Meng, Xiang Pan, Huiying Shi, Han Zhu, Xiaozhong Xu, Lei Sun, Zhenzhong Chen, Shan Liu, Fangyuan Kong, Haotian Fan, Yifang Xu, Haoran Xu, Mengduo Yang, Jie Zhou, Jiaze Li, Shijie Wen, Mai Xu, Da Li, Shunyu Yao, Jiazhi Du, Wangmeng Zuo, Zhibo Li, Shuai He, Anlong Ming, Huiyuan Fu, Huadong Ma, Yong Wu, Fie Xue, Guozhi Zhao, Lina Du, Jie Guo, Yu Zhang, Huimin Zheng, Junhao Chen, Yue Liu, Dulan Zhou, Kele Xu, Qisheng Xu, Tao Sun, Zhixiang Ding, Yuhang Hu
TL;DR
The paper documents the NTIRE 2024 Challenge on Short-form UGC Video Quality Assessment and introduces KVQ, a large, realistic S-UGC benchmark derived from Kwai/Kuaishou with diverse content and degradation workflows. It establishes an evaluation protocol combining SROCC, PLCC, and fine-grained ranking metrics, and reports a two-phase competition that attracted numerous teams and predictions. Across the submissions, top systems deploy a spectrum of architectures—from multi-task and ensemble pipelines to self-supervised priors and cross-modal fusion—achieving state-of-the-art performance (PLCC/SROCC > 0.9) on KVQ and surpassing traditional baselines. The study provides a rich benchmark and actionable insights into effective S-UGC VQA strategies, with practical implications for quality control in short-form video platforms and future research directions in perceptual video quality assessment.
Abstract
This paper reviews the NTIRE 2024 Challenge on Shortform UGC Video Quality Assessment (S-UGC VQA), where various excellent solutions are submitted and evaluated on the collected dataset KVQ from popular short-form video platform, i.e., Kuaishou/Kwai Platform. The KVQ database is divided into three parts, including 2926 videos for training, 420 videos for validation, and 854 videos for testing. The purpose is to build new benchmarks and advance the development of S-UGC VQA. The competition had 200 participants and 13 teams submitted valid solutions for the final testing phase. The proposed solutions achieved state-of-the-art performances for S-UGC VQA. The project can be found at https://github.com/lixinustc/KVQChallenge-CVPR-NTIRE2024.