Fine-grained Image Quality Assessment for Perceptual Image Restoration

TL;DR

This work addresses the mismatch between traditional IQA metrics and perceptual restoration quality by introducing FGRestore, a large dataset capturing subtle restoration quality differences across six tasks, with MOS and 30,886 pairwise preferences. Building on FGRestore, FGResQ combines degradation-aware feature learning with a dual-branch predictor to deliver both absolute quality scores and pairwise rankings. The degradation-aware module leverages CLIP-based semantic alignment and a bidirectional contrastive loss, while the prediction heads enable unified evaluation across restoration tasks. The results underscore the inadequacy of score-based IQA in IR and demonstrate the practical value of a fine-grained evaluation framework for guiding restoration development and benchmarking.

Abstract

Recent years have witnessed remarkable achievements in perceptual image restoration (IR), creating an urgent demand for accurate image quality assessment (IQA), which is essential for both performance comparison and algorithm optimization. Unfortunately, the existing IQA metrics exhibit inherent weakness for IR task, particularly when distinguishing fine-grained quality differences among restored images. To address this dilemma, we contribute the first-of-its-kind fine-grained image quality assessment dataset for image restoration, termed FGRestore, comprising 18,408 restored images across six common IR tasks. Beyond conventional scalar quality scores, FGRestore was also annotated with 30,886 fine-grained pairwise preferences. Based on FGRestore, a comprehensive benchmark was conducted on the existing IQA metrics, which reveal significant inconsistencies between score-based IQA evaluations and the fine-grained restoration quality. Motivated by these findings, we further propose FGResQ, a new IQA model specifically designed for image restoration, which features both coarse-grained score regression and fine-grained quality ranking. Extensive experiments and comparisons demonstrate that FGResQ significantly outperforms state-of-the-art IQA metrics. Codes and model weights have been released in https://sxfly99.github.io/FGResQ-Homepage.

Figures (18)

• Figure 1: Illustration of the fine-grained challenge in IQA for image restoration. Both IR algorithm comparison and optimization processes require distinguishing subtle quality differences between restored images. Existing IQA metrics fail to provide correct rankings for fine-grained image pairs. (Best viewed zoomed in.)
• Figure 2: Overview of our method. (a) FGRestore provides comprehensive fine-grained quality annotations for multiple IR tasks. (b) FGResQ enables both coarse-grained quality scoring and fine-grained quality ranking capabilities.
• Figure 3: Overview of the FGRestore dataset construction.
• Figure 4: Consistency analysis between MOS scores and human preference rankings. Point sizes represent frequency of image pairs. Red regions indicate inconsistent cases where MOS scores and human preferences disagree.
• Figure 5: Overview of the proposed FGResQ framework.