Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Emphasizing Crucial Features for Efficient Image Restoration

Hu Gao, Bowen Ma, Ying Zhang, Jingfan Yang, Jing Yang, Depeng Dang

TL;DR

This work tackles image restoration when degradation varies across regions by introducing ECFNet, a CNN-based U-Net variant that integrates a spatial-frequency attention mechanism (SFAM) and a three-branch multi-scale block (MSBlock). SFAM decomposes into SDAM, which locates degradation through spatial and channel cues, and FDAM, which amplifies high-frequency information to highlight spectral differences between sharp and degraded regions; MSBlock enables global dependencies at high resolution, while subsequent scales use MSSFBlocks for efficiency. The model is trained with a multi-term loss that combines reconstruction, edge, and frequency-domain terms, and is validated across three tasks—image dehazing, defocus deblurring, and desnowing—showing state-of-the-art performance with substantial efficiency gains on both synthetic and real-world datasets. The findings demonstrate the practical potential of region-aware restoration and multi-scale CNN architectures for robust, real-time image restoration in diverse environments.

Abstract

Image restoration is a challenging ill-posed problem which estimates latent sharp image from its degraded counterpart. Although the existing methods have achieved promising performance by designing novelty architecture of module, they ignore the fact that different regions in a corrupted image undergo varying degrees of degradation. In this paper, we propose an efficient and effective framework to adapt to varying degrees of degradation across different regions for image restoration. Specifically, we design a spatial and frequency attention mechanism (SFAM) to emphasize crucial features for restoration. SFAM consists of two modules: the spatial domain attention module (SDAM) and the frequency domain attention module (FDAM). The SFAM discerns the degradation location through spatial selective attention and channel selective attention in the spatial domain, while the FDAM enhances high-frequency signals to amplify the disparities between sharp and degraded image pairs in the spectral domain. Additionally, to capture global range information, we introduce a multi-scale block (MSBlock) that consists of three scale branches, each containing multiple simplified channel attention blocks (SCABlocks) and a multi-scale feed-forward block (MSFBlock). Finally, we propose our ECFNet, which integrates the aforementioned components into a U-shaped backbone for recovering high-quality images. Extensive experimental results demonstrate the effectiveness of ECFNet, outperforming state-of-the-art (SOTA) methods on both synthetic and real-world datasets.

Emphasizing Crucial Features for Efficient Image Restoration

TL;DR

This work tackles image restoration when degradation varies across regions by introducing ECFNet, a CNN-based U-Net variant that integrates a spatial-frequency attention mechanism (SFAM) and a three-branch multi-scale block (MSBlock). SFAM decomposes into SDAM, which locates degradation through spatial and channel cues, and FDAM, which amplifies high-frequency information to highlight spectral differences between sharp and degraded regions; MSBlock enables global dependencies at high resolution, while subsequent scales use MSSFBlocks for efficiency. The model is trained with a multi-term loss that combines reconstruction, edge, and frequency-domain terms, and is validated across three tasks—image dehazing, defocus deblurring, and desnowing—showing state-of-the-art performance with substantial efficiency gains on both synthetic and real-world datasets. The findings demonstrate the practical potential of region-aware restoration and multi-scale CNN architectures for robust, real-time image restoration in diverse environments.

Abstract

Image restoration is a challenging ill-posed problem which estimates latent sharp image from its degraded counterpart. Although the existing methods have achieved promising performance by designing novelty architecture of module, they ignore the fact that different regions in a corrupted image undergo varying degrees of degradation. In this paper, we propose an efficient and effective framework to adapt to varying degrees of degradation across different regions for image restoration. Specifically, we design a spatial and frequency attention mechanism (SFAM) to emphasize crucial features for restoration. SFAM consists of two modules: the spatial domain attention module (SDAM) and the frequency domain attention module (FDAM). The SFAM discerns the degradation location through spatial selective attention and channel selective attention in the spatial domain, while the FDAM enhances high-frequency signals to amplify the disparities between sharp and degraded image pairs in the spectral domain. Additionally, to capture global range information, we introduce a multi-scale block (MSBlock) that consists of three scale branches, each containing multiple simplified channel attention blocks (SCABlocks) and a multi-scale feed-forward block (MSFBlock). Finally, we propose our ECFNet, which integrates the aforementioned components into a U-shaped backbone for recovering high-quality images. Extensive experimental results demonstrate the effectiveness of ECFNet, outperforming state-of-the-art (SOTA) methods on both synthetic and real-world datasets.
Paper Structure (22 sections, 14 equations, 12 figures, 10 tables)

This paper contains 22 sections, 14 equations, 12 figures, 10 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Image Restoration
  4. CNN-based models
  5. Transformer-based models
  6. MLP-based models
  7. Frequency Learning
  8. Method
  9. Overall Pipeline
  10. Spatial and Frequency Attention Mechanism (SFAM)
  11. Multi-scale Block (MSBlock)
  12. Loss Function
  13. Experiments
  14. Experimental Settings
  15. Datasets
  16. ...and 7 more sections

Figures (12)

  • Figure 1: Computational cost vs. PSNR between our ECFNet and other state-of-the-art algorithms. Left: Image defocus deblurring on the DDPD dataset DPDNet, our ECFNet achieve the SOTA performance with up to 57.9% of cost reduction. Right: Image dehazing on the SOTS dataset SOTli2018benchmarking, our ECFNet achieve the SOTA performance with up to 84.8% of cost reduction.
  • Figure 2: Varying degrees of degradation across different regions. On the left is an image affected by snow from Snow100k desnownet, in the middle is an image affected by blurring from DPDD DPDNet, and on the right is an image affected by haze from O-Haze ohazeancuti2018haze. The yellow box represents the most heavily degrade area, followed by the red box, while the blue box indicates the least degrade region.
  • Figure 3: (a) Overall architecture of the proposed ECFNet. (b) Multi-scale spatial feature blocks (MSSFBlock). (c) ConvS extracts the shallow features for low-resolution degraded images. (d) Multi-scale block (MSBlock), comprising three scale branches.
  • Figure 4: (a) The spatial and frequency attention mechanism (SFAM) that contains: (b) the spatial domain attention module (SDAM) and (c) the frequency domain attention module (FDAM).
  • Figure 5: Image dehazing comparisons on the SOTS dataset SOTli2018benchmarking. The top image is obtained from SOTS-Indoor while the bottom one is from SOTS-Outdoor.
  • ...and 7 more figures