Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Perception- and Fidelity-aware Reduced-Reference Super-Resolution Image Quality Assessment

Xinying Lin, Xuyang Liu, Hong Yang, Xiaohai He, Honggang Chen

TL;DR

This paper proposes a novel dual-branch reduced-reference SR-IQA network, i.e., Perception- and Fidelity-aware SR-IQA (PFIQA), which substantially aligns with the human visual system, enabling a comprehensive SR image evaluation.

Abstract

With the advent of image super-resolution (SR) algorithms, how to evaluate the quality of generated SR images has become an urgent task. Although full-reference methods perform well in SR image quality assessment (SR-IQA), their reliance on high-resolution (HR) images limits their practical applicability. Leveraging available reconstruction information as much as possible for SR-IQA, such as low-resolution (LR) images and the scale factors, is a promising way to enhance assessment performance for SR-IQA without HR for reference. In this letter, we attempt to evaluate the perceptual quality and reconstruction fidelity of SR images considering LR images and scale factors. Specifically, we propose a novel dual-branch reduced-reference SR-IQA network, \ie, Perception- and Fidelity-aware SR-IQA (PFIQA). The perception-aware branch evaluates the perceptual quality of SR images by leveraging the merits of global modeling of Vision Transformer (ViT) and local relation of ResNet, and incorporating the scale factor to enable comprehensive visual perception. Meanwhile, the fidelity-aware branch assesses the reconstruction fidelity between LR and SR images through their visual perception. The combination of the two branches substantially aligns with the human visual system, enabling a comprehensive SR image evaluation. Experimental results indicate that our PFIQA outperforms current state-of-the-art models across three widely-used SR-IQA benchmarks. Notably, PFIQA excels in assessing the quality of real-world SR images.

Perception- and Fidelity-aware Reduced-Reference Super-Resolution Image Quality Assessment

TL;DR

This paper proposes a novel dual-branch reduced-reference SR-IQA network, i.e., Perception- and Fidelity-aware SR-IQA (PFIQA), which substantially aligns with the human visual system, enabling a comprehensive SR image evaluation.

Abstract

With the advent of image super-resolution (SR) algorithms, how to evaluate the quality of generated SR images has become an urgent task. Although full-reference methods perform well in SR image quality assessment (SR-IQA), their reliance on high-resolution (HR) images limits their practical applicability. Leveraging available reconstruction information as much as possible for SR-IQA, such as low-resolution (LR) images and the scale factors, is a promising way to enhance assessment performance for SR-IQA without HR for reference. In this letter, we attempt to evaluate the perceptual quality and reconstruction fidelity of SR images considering LR images and scale factors. Specifically, we propose a novel dual-branch reduced-reference SR-IQA network, \ie, Perception- and Fidelity-aware SR-IQA (PFIQA). The perception-aware branch evaluates the perceptual quality of SR images by leveraging the merits of global modeling of Vision Transformer (ViT) and local relation of ResNet, and incorporating the scale factor to enable comprehensive visual perception. Meanwhile, the fidelity-aware branch assesses the reconstruction fidelity between LR and SR images through their visual perception. The combination of the two branches substantially aligns with the human visual system, enabling a comprehensive SR image evaluation. Experimental results indicate that our PFIQA outperforms current state-of-the-art models across three widely-used SR-IQA benchmarks. Notably, PFIQA excels in assessing the quality of real-world SR images.
Paper Structure (13 sections, 2 equations, 3 figures, 5 tables)

This paper contains 13 sections, 2 equations, 3 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Image Quality Assessment
  4. Super-Resolution Image Quality Assessment
  5. Methodology
  6. Feature Extraction
  7. Adaptive Fusion
  8. Patch-Weighted Quality Regression
  9. Experiments
  10. Experimental Settings
  11. Main Results
  12. Ablation Study
  13. Conclusion

Figures (3)

  • Figure 1: Overview of proposed PFIQA method. This framework consists of Perception-aware Assessment Branch (PA Branch) and Fidelity-aware Assessment Branch (FA Branch) for SR-IQA.
  • Figure 2: Examples of predicted scores on RealSRQjiang2022single include DISQzhao2021learning (Previous SR-IQA SOTA) / PFIQA (Ours) / MOS. We consistently crop and enlarge each image for better visualization. The images are arranged in order of increasing image quality, with corresponding MOS values progressively increasing.
  • Figure 3: Scatter plots of ground-truth mean opinion scores (MOSs) against predicted scores of six generic IQA methods, two SR-IQA methods and proposed PFIQA on RealSRQjiang2022single dataset. The MOS values have been scaled to range between 0 and 1. Blue points represent the results of the corresponding methods, and the linear fitting of all points is marked by a red straight line.