REQA: Coarse-to-fine Assessment of Image Quality to Alleviate the Range Effect

TL;DR

The proposed method can alleviate the range effect compared to the state-of-the-art methods effectively and design a feedback network to conduct the coarse-to-fine assessment.

Abstract

Blind image quality assessment (BIQA) of user generated content (UGC) suffers from the range effect which indicates that on the overall quality range, mean opinion score (MOS) and predicted MOS (pMOS) are well correlated; focusing on a particular range, the correlation is lower. The reason for the range effect is that the predicted deviations both in a wide range and in a narrow range destroy the uniformity between MOS and pMOS. To tackle this problem, a novel method is proposed from coarse-grained metric to fine-grained prediction. Firstly, we design a rank-and-gradient loss for coarse-grained metric. The loss keeps the order and grad consistency between pMOS and MOS, thereby reducing the predicted deviation in a wide range. Secondly, we propose multi-level tolerance loss to make fine-grained prediction. The loss is constrained by a decreasing threshold to limite the predicted deviation in narrower and narrower ranges. Finally, we design a feedback network to conduct the coarse-to-fine assessment. On the one hand, the network adopts feedback blocks to process multi-scale distortion features iteratively and on the other hand, it fuses non-local context feature to the output of each iteration to acquire more quality-aware feature representation. Experimental results demonstrate that the proposed method can alleviate the range effect compared to the state-of-the-art methods effectively.

Figures (6)

• Figure 1: Visualization of the predicted results of HyperIQA HyperIQA on CLIVELIVEC. It depicts the range effect: In terms of the overall quality range, MOS and predicted MOS seem to be well correlated; While focusing on a particular range (e.g., points within the green box), the correlation is low.
• Figure 2: Flowchart of the proposed REQA. REQA adopts a feedback hierarchy to realize coarse-to-fine quality assessment within multiple time steps. Here, a rank-and-gradient metric accomplishes coarse-grained assessment, and multi-stage pMOS refinements complete fine-grained assessment. In addition, REQA integrates image feature representations from two aspects, where the first is to process multi-scale distortion features through iteration and the second is the fusion of context features from Transfomer EncoderTE.
• Figure 3: The overview framework of REQA. It consists of three parts as the baseline network:ResNet-50
• Figure 4: Architecture details of feedback block.
• Figure 5: Visualization of predicted results. Green boxes focuses on the particular range and red boxes concerns the outliers .