Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

PromptIQA: Boosting the Performance and Generalization for No-Reference Image Quality Assessment via Prompts

Zewen Chen, Haina Qin, Juan Wang, Chunfeng Yuan, Bing Li, Weiming Hu, Liang Wang

TL;DR

This work proposes a Prompt-based IQA (PromptIQA) that can directly adapt to new requirements without fine-tuning after training, and outperforms SOTA methods with higher performance and better generalization.

Abstract

Due to the diversity of assessment requirements in various application scenarios for the IQA task, existing IQA methods struggle to directly adapt to these varied requirements after training. Thus, when facing new requirements, a typical approach is fine-tuning these models on datasets specifically created for those requirements. However, it is time-consuming to establish IQA datasets. In this work, we propose a Prompt-based IQA (PromptIQA) that can directly adapt to new requirements without fine-tuning after training. On one hand, it utilizes a short sequence of Image-Score Pairs (ISP) as prompts for targeted predictions, which significantly reduces the dependency on the data requirements. On the other hand, PromptIQA is trained on a mixed dataset with two proposed data augmentation strategies to learn diverse requirements, thus enabling it to effectively adapt to new requirements. Experiments indicate that the PromptIQA outperforms SOTA methods with higher performance and better generalization. The code will be available.

PromptIQA: Boosting the Performance and Generalization for No-Reference Image Quality Assessment via Prompts

TL;DR

This work proposes a Prompt-based IQA (PromptIQA) that can directly adapt to new requirements without fine-tuning after training, and outperforms SOTA methods with higher performance and better generalization.

Abstract

Due to the diversity of assessment requirements in various application scenarios for the IQA task, existing IQA methods struggle to directly adapt to these varied requirements after training. Thus, when facing new requirements, a typical approach is fine-tuning these models on datasets specifically created for those requirements. However, it is time-consuming to establish IQA datasets. In this work, we propose a Prompt-based IQA (PromptIQA) that can directly adapt to new requirements without fine-tuning after training. On one hand, it utilizes a short sequence of Image-Score Pairs (ISP) as prompts for targeted predictions, which significantly reduces the dependency on the data requirements. On the other hand, PromptIQA is trained on a mixed dataset with two proposed data augmentation strategies to learn diverse requirements, thus enabling it to effectively adapt to new requirements. Experiments indicate that the PromptIQA outperforms SOTA methods with higher performance and better generalization. The code will be available.
Paper Structure (14 sections, 4 equations, 3 figures, 6 tables)

This paper contains 14 sections, 4 equations, 3 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. No-Reference Image Quality Assessment
  4. Mixed Training For Image Quality Assessment
  5. The Proposed Method
  6. Problem Formulation
  7. Overview of the PromptIQA
  8. Experiments
  9. Experiments Settings
  10. Comparisons With State-Of-The-Arts
  11. Generalization Comparison on Other Assessment Requirements
  12. Prompt Discussion
  13. Ablation Study
  14. Conclusion

Figures (3)

  • Figure 1: Comparison of the typical and prompt-based IQA frameworks. For a dataset $M$, pairs of images and corresponding scores $P_i^M$ constitute the ISP prompts, which represent the assessment requirement for the dataset $M$.
  • Figure 2: Overview of our proposed PromptIQA.
  • Figure 3: The impact of the number of ISPs in an ISPP on PromptIQA performance.