Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Deep Bi-directional Attention Network for Image Super-Resolution Quality Assessment

Yixiao Li, Xiaoyuan Yang, Jun Fu, Guanghui Yue, Wei Zhou

TL;DR

A deep Bidirectional Attention Network (BiAtten-Net) is constructed that dynamically deepens visual attention to distortions in both processes, which aligns well with the human visual system (HVS).

Abstract

There has emerged a growing interest in exploring efficient quality assessment algorithms for image super-resolution (SR). However, employing deep learning techniques, especially dual-branch algorithms, to automatically evaluate the visual quality of SR images remains challenging. Existing SR image quality assessment (IQA) metrics based on two-stream networks lack interactions between branches. To address this, we propose a novel full-reference IQA (FR-IQA) method for SR images. Specifically, producing SR images and evaluating how close the SR images are to the corresponding HR references are separate processes. Based on this consideration, we construct a deep Bi-directional Attention Network (BiAtten-Net) that dynamically deepens visual attention to distortions in both processes, which aligns well with the human visual system (HVS). Experiments on public SR quality databases demonstrate the superiority of our proposed BiAtten-Net over state-of-the-art quality assessment methods. In addition, the visualization results and ablation study show the effectiveness of bi-directional attention.

Deep Bi-directional Attention Network for Image Super-Resolution Quality Assessment

TL;DR

A deep Bidirectional Attention Network (BiAtten-Net) is constructed that dynamically deepens visual attention to distortions in both processes, which aligns well with the human visual system (HVS).

Abstract

There has emerged a growing interest in exploring efficient quality assessment algorithms for image super-resolution (SR). However, employing deep learning techniques, especially dual-branch algorithms, to automatically evaluate the visual quality of SR images remains challenging. Existing SR image quality assessment (IQA) metrics based on two-stream networks lack interactions between branches. To address this, we propose a novel full-reference IQA (FR-IQA) method for SR images. Specifically, producing SR images and evaluating how close the SR images are to the corresponding HR references are separate processes. Based on this consideration, we construct a deep Bi-directional Attention Network (BiAtten-Net) that dynamically deepens visual attention to distortions in both processes, which aligns well with the human visual system (HVS). Experiments on public SR quality databases demonstrate the superiority of our proposed BiAtten-Net over state-of-the-art quality assessment methods. In addition, the visualization results and ablation study show the effectiveness of bi-directional attention.
Paper Structure (7 sections, 4 equations, 3 figures, 2 tables)

This paper contains 7 sections, 4 equations, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Proposed Method
  3. Validation
  4. Experimental Protocols
  5. Performance Comparisons
  6. Ablation Study
  7. Conclusion

Figures (3)

  • Figure 1: The overall framework of the proposed BiAtten-Net. Encoder indicates the stacking of a convolution layer, a batch normalization layer, and the activation function (ReLU). Conv refers to convolution layer, and the kernel size of all Convs is $3\times 3$. Pool represents the adaptive average pooling layer, and Atten map is the attention map. $+$ refers to Shortcut, which directly adds the inputs of BAB to the end of the block.
  • Figure 2: Visualization comparisons of feature maps regarding the proposed bi-directional attention block. Brick HR image and Flower HR image are HR references. Brick SR image and Flower SR image are SR images. The remaining images are feature maps before and after BAB in two branches.
  • Figure 3: Similarity of “$X_{1}\& X_{2}$”, where $X_{1}$ is initial HR reference or SR image, $X_{2}$ is the feature map before or after BAB. $(B)$ and $(F)$ denote “Brick” and “Flower”, respectively. The higher the SSIM, the more similar the two images are.