A Perceptual Quality Assessment Exploration for AIGC Images

TL;DR

This work addresses perceptual quality assessment for AI-generated images (AGIs) by introducing AGIQA-1K, the first large-scale, perception-labeled database of 1,080 diffusion-generated AGIs. It details a rigorous subjective study following ITU-BT.500-13 guidelines to capture five quality aspects and benchmarks a range of no-reference IQA methods on AGIs. The findings show current NR-IQA approaches struggle to predict AGI quality, with performance drops on more diverse diffusion models, underscoring the need for AGI-specific perceptual quality models. Overall, the paper provides both a valuable dataset and a benchmarking framework to propel future AGI perceptual quality research.

Abstract

\underline{AI} \underline{G}enerated \underline{C}ontent (\textbf{AIGC}) has gained widespread attention with the increasing efficiency of deep learning in content creation. AIGC, created with the assistance of artificial intelligence technology, includes various forms of content, among which the AI-generated images (AGIs) have brought significant impact to society and have been applied to various fields such as entertainment, education, social media, etc. However, due to hardware limitations and technical proficiency, the quality of AIGC images (AGIs) varies, necessitating refinement and filtering before practical use. Consequently, there is an urgent need for developing objective models to assess the quality of AGIs. Unfortunately, no research has been carried out to investigate the perceptual quality assessment for AGIs specifically. Therefore, in this paper, we first discuss the major evaluation aspects such as technical issues, AI artifacts, unnaturalness, discrepancy, and aesthetics for AGI quality assessment. Then we present the first perceptual AGI quality assessment database, AGIQA-1K, which consists of 1,080 AGIs generated from diffusion models. A well-organized subjective experiment is followed to collect the quality labels of the AGIs. Finally, we conduct a benchmark experiment to evaluate the performance of current image quality assessment (IQA) models.

Figures (6)

• Figure 1: Illustration of the generation process of AGIs and NSIs, where NSIs are captured from the natural scenes and AGIs are directly generated from AI models.
• Figure 2: Sample images from the AGIQA-1k database, where the first to sixth rows show AGIs with (bird, cat, batman, kid, man, woman) as the main objects respectively.
• Figure 3: The normalized probability distributions of the quality-related attributes for NSIs and AGIs. The distributions are obtained from 10,073 NSIs in the KonIQ-10k IQA database koniq10k and 1,080 AGIs in the proposed AGIQA-1k database respectively. The 'color' indicates the colorfulness of the images and the 'SI' (spatial information) stands for the content diversity of the images.
• Figure 4: Exhibition for some common AGI distortions, where the generation keywords are marked in the top right. The content of (a) is poorly visible due to the blur and inexplicit texture. Unexpected artifacts are introduced to the bottom left of (b). (c) contains unnatural content such as the hands of the woman do not cope with common sense. (d) is too simple and does not fit the text keyword of "Wearing Coat".
• Figure 5: An example of the quality assessment interface, where the AGI and corresponding keywords are shown at the same time. The subject can then evaluate the quality of AGIs and record the quality scores with the scroll bar on the right.