Progressive Feature Fusion Network for Enhancing Image Quality Assessment

TL;DR

Experimental results show that compared with the current mainstream image quality assessment methods, the proposed network can achieve more accurate image quality assessment and ranks second in the benchmark of Challenge on Learned Image Compression in the image perceptual model track.

Abstract

Image compression has been applied in the fields of image storage and video broadcasting. However, it's formidably tough to distinguish the subtle quality differences between those distorted images generated by different algorithms. In this paper, we propose a new image quality assessment framework to decide which image is better in an image group. To capture the subtle differences, a fine-grained network is adopted to acquire multi-scale features. Subsequently, we design a cross subtract block for separating and gathering the information within positive and negative image pairs. Enabling image comparison in feature space. After that, a progressive feature fusion block is designed, which fuses multi-scale features in a novel progressive way. Hierarchical spatial 2D features can thus be processed gradually. Experimental results show that compared with the current mainstream image quality assessment methods, the proposed network can achieve more accurate image quality assessment and ranks second in the benchmark of CLIC in the image perceptual model track.

Figures (3)

• Figure 1: PRFNet Architecture. The input of the network includes the reference image $O$ and two distorted images $A$ and $B$ with different degrees of compression. In the feature extraction module, feature maps of the three images at four scales ($F_{disa}^i$,$F_{ref}^i$,$F_{disb}^i$, where $i= 1, 2, 3, 4$) can be obtained respectively. Instead of measuring the differences between the compressed and reference image in low level space, cross subtraction blocks utilize the differences in the features. A progressive feature fusion block is set to fuse different scales of features progressively. Finally, multi-layer perceptron (MLP) networks are used to get the classification result.
• Figure 2: The blocks of (a)the ResNet bottleneck, (b)the Res2Net block, and (c)the SE_Res2Net block. The receptive field of a traditional ResNet bottleneck can be relatively narrow, while Res2Net can expand the receptive field by decomposing and recomposing. SE_Res2Net block can extract more representative features as a SE_block is followed to grasp the global information.
• Figure 3: (a)Cross Subtraction Block. This block calculates the differences of feature maps of image $A$, $B$, and $O$, and carries out the feature difference maps by subtraction and cross operations. (b)PRogressive feature Fusion Block. In this block, weights are obtained by a series of operations on deep-level features, then fused with lower-level features by hadamard product.