Dual-Representation Interaction Driven Image Quality Assessment with Restoration Assistance

TL;DR

The DRI method is introduced to obtain degradation vectors and quality vectors of images, which separately model the degradation and quality information of low-quality images, and the Representation-based Semantic Loss (RS Loss) is designed to assist in enhancing effective interaction between representations.

Abstract

No-Reference Image Quality Assessment for distorted images has always been a challenging problem due to image content variance and distortion diversity. Previous IQA models mostly encode explicit single-quality features of synthetic images to obtain quality-aware representations for quality score prediction. However, performance decreases when facing real-world distortion and restored images from restoration models. The reason is that they do not consider the degradation factors of the low-quality images adequately. To address this issue, we first introduce the DRI method to obtain degradation vectors and quality vectors of images, which separately model the degradation and quality information of low-quality images. After that, we add the restoration network to provide the MOS score predictor with degradation information. Then, we design the Representation-based Semantic Loss (RS Loss) to assist in enhancing effective interaction between representations. Extensive experimental results demonstrate that the proposed method performs favorably against existing state-of-the-art models on both synthetic and real-world datasets.

Figures (6)

• Figure 1: (a) The conventional pipeline QPT to train a quality-aware representation encoder, the blue and green balls represent the Quality-Positive and Negative samples, respectively. (b) Our Dual-Representation Interaction method adds Degradation-Positive samples symbolized by yellow balls. We also propose Representation-based Semantic Loss (RS Loss) to constraint model training.
• Figure 2: Overview of our two-stage Dual-Representation Interaction IQA (DRI-IQA) architecture. Stage 1 is set to train the Dual-Representation Extractor Net (DRE-Net), extracting quality representation and degradation representation simultaneously and reserving the features for guiding the Restoration Network. The modules in the blue region are the components we use in stage 1. In Stage 2, we train the score predictor with the assistance of the Restoration Network, that is the complete network joins in training. Meanwhile, the Representation-based Semantic Loss (RS Loss) is leveraged to promote implicit interaction between two representations and enhance the performance of the Restoration Network.
• Figure 3: The $x_{\text{1}}$ and $x_{\text{2}}$ are generated from the same input images with varied degradation distortion, while $y$ is the input image with both different content and distortion. We get two patches randomly on each of the three images and put them into the Dual-Representation Extractor (DRE) to obtain the features. The table shows the choice strategy for the positive and negative samples of the two halves of the features.
• Figure 4: Details of our guidance strategy. The cuboid in orange and gray represent the quality-aware part of Dual-Representation and the features encoded from input images, respectively. Also, the direction of the cuboid represents the transposed form. TGB denotes Transposed Guidance Block and STB represents Swin Transformer Block MANIQA.
• Figure 5: Predicted MOS scores by our DRI-IQA model and the MOS score labels on low-quality images with varying degradations.