Quality Assessment for AI Generated Images with Instruction Tuning

TL;DR

This work tackles the challenge of evaluating human preferences for AI-generated images by introducing AIGCIQA2023+, a large-scale IQA dataset annotated along three dimensions: quality, authenticity, and text-image correspondence. It then presents MINT-IQA, a multimodal framework that uses prompt segmentation, cross-modal Q-Formers, and instruction tuning to predict multi-perspective quality scores and generate explanatory VQA-style outputs. The model achieves state-of-the-art performance on AIGC IQA benchmarks and strong results on traditional IQA datasets, while its instruction-tuned explanations enhance interpretability and potential feedback for improving generation methods. The combination of a rich, multidimensional dataset and an explainable, instruction-tuned model offers a practical path for QoE-aware QA and improvement loops in AI-generated imagery.

Abstract

Artificial Intelligence Generated Content (AIGC) has grown rapidly in recent years, among which AI-based image generation has gained widespread attention due to its efficient and imaginative image creation ability. However, AI-generated Images (AIGIs) may not satisfy human preferences due to their unique distortions, which highlights the necessity to understand and evaluate human preferences for AIGIs. To this end, in this paper, we first establish a novel Image Quality Assessment (IQA) database for AIGIs, termed AIGCIQA2023+, which provides human visual preference scores and detailed preference explanations from three perspectives including quality, authenticity, and correspondence. Then, based on the constructed AIGCIQA2023+ database, this paper presents a MINT-IQA model to evaluate and explain human preferences for AIGIs from Multi-perspectives with INstruction Tuning. Specifically, the MINT-IQA model first learn and evaluate human preferences for AI-generated Images from multi-perspectives, then via the vision-language instruction tuning strategy, MINT-IQA attains powerful understanding and explanation ability for human visual preference on AIGIs, which can be used for feedback to further improve the assessment capabilities. Extensive experimental results demonstrate that the proposed MINT-IQA model achieves state-of-the-art performance in understanding and evaluating human visual preferences for AIGIs, and the proposed model also achieves competing results on traditional IQA tasks compared with state-of-the-art IQA models. The AIGCIQA2023+ database and MINT-IQA model are available at: https://github.com/IntMeGroup/MINT-IQA.

Figures (11)

• Figure 1: We present AIGCIQA2023+, a large-scale dataset that includes 2400 images generated from 100 prompts, 7200 MOSs from 3 perspectives, and 16,800 fine-grained preference-explanation annotations.
• Figure 2: Examples generated by our MINT-IQA model, demonstrating its diverse capabilities including image quality assessment from multiple perspectives, abundant degradation-aware visual question answering, comprehensive human visual preference explaining, etc.
• Figure 3: Rating comparisons between three perspectives. (a) The quality score of the left image is higher, but other two scores are lower. (b) The authenticity score of the left image is higher, but other two scores are lower. (c) The correspondence score of the left image is higher, but other two scores are lower.
• Figure 4: Illustration of the subjective assessment interface. The subjects are instructed to make fine-grained assessments and annotations by clicking the checkboxes and give further detailed explanations by inputting text descriptions.
• Figure 5: Illustration of the generated visual question-answering pairs for instruction tuning.