Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AMSA-UNet: An Asymmetric Multiple Scales U-net Based on Self-attention for Deblurring

Yingying Wang

TL;DR

This work tackles spatial information loss and limited long-range modeling in single-scale U-Nets for image deblurring by introducing AMSA-UNet, an asymmetric multi-scale U-Net with self-attention in the decoder and FFT-based speedups. The architecture fuses multi-scale features through AFF, leverages a DFFN with a learnable quantization matrix $W$ in both encoder and decoder paths, and employs a frequency-domain self-attention solver (FSAS) to reduce complexity from $O(n^2)$-style computations. The method demonstrates superior PSNR/SSIM and faster inference on GoPro and Kohler datasets, outperforming eight baselines and showing strong generalization. Overall, AMSA-UNet achieves high deblurring accuracy with reduced computational demands, making it suitable for practical, real-time applications in diverse blurred imagery where long-range dependencies matter.

Abstract

The traditional ingle-scale U-Net often leads to the loss of spatial information during deblurring, which affects the deblurring accracy. Additionally, due to the convolutional method's limitation in capturing long-range dependencies, the quality of the recovered image is degraded. To address the above problems, an asymmetric multiple scales U-net based on self-attention (AMSA-UNet) is proposed to improve the accuracy and computational complexity. By introducing a multiple-scales U shape architecture, the network can focus on blurry regions at the global level and better recover image details at the local level. In order to overcome the limitations of traditional convolutional methods in capturing the long-range dependencies of information, a self-attention mechanism is introduced into the decoder part of the backbone network, which significantly increases the model's receptive field, enabling it to pay more attention to semantic information of the image, thereby producing more accurate and visually pleasing deblurred images. What's more, a frequency domain-based computation method was introduced to reduces the computation amount. The experimental results demonstrate that the proposed method exhibits significant improvements in both accuracy and speed compared to eight excellent methods

AMSA-UNet: An Asymmetric Multiple Scales U-net Based on Self-attention for Deblurring

TL;DR

This work tackles spatial information loss and limited long-range modeling in single-scale U-Nets for image deblurring by introducing AMSA-UNet, an asymmetric multi-scale U-Net with self-attention in the decoder and FFT-based speedups. The architecture fuses multi-scale features through AFF, leverages a DFFN with a learnable quantization matrix in both encoder and decoder paths, and employs a frequency-domain self-attention solver (FSAS) to reduce complexity from -style computations. The method demonstrates superior PSNR/SSIM and faster inference on GoPro and Kohler datasets, outperforming eight baselines and showing strong generalization. Overall, AMSA-UNet achieves high deblurring accuracy with reduced computational demands, making it suitable for practical, real-time applications in diverse blurred imagery where long-range dependencies matter.

Abstract

The traditional ingle-scale U-Net often leads to the loss of spatial information during deblurring, which affects the deblurring accracy. Additionally, due to the convolutional method's limitation in capturing long-range dependencies, the quality of the recovered image is degraded. To address the above problems, an asymmetric multiple scales U-net based on self-attention (AMSA-UNet) is proposed to improve the accuracy and computational complexity. By introducing a multiple-scales U shape architecture, the network can focus on blurry regions at the global level and better recover image details at the local level. In order to overcome the limitations of traditional convolutional methods in capturing the long-range dependencies of information, a self-attention mechanism is introduced into the decoder part of the backbone network, which significantly increases the model's receptive field, enabling it to pay more attention to semantic information of the image, thereby producing more accurate and visually pleasing deblurred images. What's more, a frequency domain-based computation method was introduced to reduces the computation amount. The experimental results demonstrate that the proposed method exhibits significant improvements in both accuracy and speed compared to eight excellent methods
Paper Structure (15 sections, 8 equations, 6 figures, 2 tables)

This paper contains 15 sections, 8 equations, 6 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Multi-scale deep convolutional network deblurring methods
  4. Transformer Deblurring Method
  5. Fourier Transform
  6. Proposed Method
  7. Encoder block
  8. Decoder block
  9. Asymmetric U-shaped network architecture
  10. Experiments
  11. Dataset and implementation details
  12. Performance comparison
  13. Ablation Experiment
  14. Conclusion
  15. Disclosure of Interests.

Figures (6)

  • Figure 1: Overall Network Architecture
  • Figure 2: The Structure of encoder block. (a) SCM, (b) FAM, (c) DFFN.
  • Figure 3: The Structure of Decoder Block. (a) AFF, (b) Fuse, (c) FSAS.
  • Figure 4: Comparison of visualization on the GoPro. For clarity, the magnified parts of the resultant images are displayed. (a) blur, (b) DeblurGAN, (c) PSS-NSC, (d) Deepdeblur, (e) proposed method, (f) ground truth
  • Figure 5: The visual results directly applied to the Kohler. For clarity, the magnified parts of the resultant images are displayed. (a) blur, (b) DeblurGAN, (c) proposed method, (d) ground truth
  • ...and 1 more figures