AIM 2024 Challenge on UHD Blind Photo Quality Assessment

TL;DR

The paper presents the AIM 2024 UHD-IQA Challenge and the UHD-IQA Benchmark, a dataset of $6073$ UHD-$1$ (4K) images with expert MOS and rich metadata, designed to stress high-resolution aesthetics and subtle degradations. It introduces eight competitive methods, spanning transformer-based multi-branch architectures, high-resolution patch strategies, multi-stage knowledge transfer, mobile teacher-student paradigms, NF-RegNets ensembles, and CLIP-based local-global scoring, all constrained to $50$ GMACs. The participating approaches combine global aesthetics, local distortions, saliency cues, and multi-scale reasoning with efficient backbones and sophisticated training strategies, achieving strong correlation and low MOS error under edge-appropriate budgets. The findings highlight practical, edge-friendly BIQA solutions for UHD imagery, with implications for photo curation, enhancement, and compression workflows.

Abstract

We introduce the AIM 2024 UHD-IQA Challenge, a competition to advance the No-Reference Image Quality Assessment (NR-IQA) task for modern, high-resolution photos. The challenge is based on the recently released UHD-IQA Benchmark Database, which comprises 6,073 UHD-1 (4K) images annotated with perceptual quality ratings from expert raters. Unlike previous NR-IQA datasets, UHD-IQA focuses on highly aesthetic photos of superior technical quality, reflecting the ever-increasing standards of digital photography. This challenge aims to develop efficient and effective NR-IQA models. Participants are tasked with creating novel architectures and training strategies to achieve high predictive performance on UHD-1 images within a computational budget of 50G MACs. This enables model deployment on edge devices and scalable processing of extensive image collections. Winners are determined based on a combination of performance metrics, including correlation measures (SRCC, PLCC, KRCC), absolute error metrics (MAE, RMSE), and computational efficiency (G MACs). To excel in this challenge, participants leverage techniques like knowledge distillation, low-precision inference, and multi-scale training. By pushing the boundaries of NR-IQA for high-resolution photos, the UHD-IQA Challenge aims to stimulate the development of practical models that can keep pace with the rapidly evolving landscape of digital photography. The innovative solutions emerging from this competition will have implications for various applications, from photo curation and enhancement to image compression.

Figures (12)

• Figure 1: Example images from the UHD-IQA dataset hosu2024uhd. They have been cropped to 64% of their original size to enhance detail visibility. The author's name from Pixabay.com is shown at the bottom right of each image.
• Figure 2: Density of quality MOS per subset. "Overall" includes all image categories, whereas "exclusive" refers to categories that are only part of the validation and test sets.
• Figure 3: Scatter plots of the predicted quality scores vs ground-truth (actual) MOS. The curves were obtained by a second-order polynomial fitting.
• Figure 4: The method proposed by the SJTU Team, using three branches sun2024assessing.
• Figure 5: Overview of the proposed GS-PIQA by Team SZU.