Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Dual-Path Coupled Image Deraining Network via Spatial-Frequency Interaction

Yuhong He, Aiwen Jiang, Lingfang Jiang, Zhifeng Wang, Lu Wang

TL;DR

The paper tackles single-image deraining by integrating spatial and frequency-domain cues through a dual-path architecture named DPCNet. It introduces Spatial Feature Extraction Block (SFEBlock) with a Spatial-Channel Transformer Block (SCTB) to fuse spatial and channel information, Frequency Feature Extraction Block (FFEBlock) based on channel-wise FFT to capture high-frequency textures, and an Adaptive Fusion Module (AFM) to fuse dual-domain features. The approach is implemented in a multi-scale encoder-decoder with Dual-Domain Blocks (DDBlock) and demonstrates state-of-the-art results on six synthetic/real benchmarks, along with improved robustness on downstream object detection tasks. The loss combines $L_1$, perceptual, and FFT terms, guiding both pixel-level restoration and frequency-domain fidelity. Overall, the work shows the value of jointly leveraging spatial and frequency information for effective deraining and practical vision applications.

Abstract

Transformers have recently emerged as a significant force in the field of image deraining. Existing image deraining methods utilize extensive research on self-attention. Though showcasing impressive results, they tend to neglect critical frequency information, as self-attention is generally less adept at capturing high-frequency details. To overcome this shortcoming, we have developed an innovative Dual-Path Coupled Deraining Network (DPCNet) that integrates information from both spatial and frequency domains through Spatial Feature Extraction Block (SFEBlock) and Frequency Feature Extraction Block (FFEBlock). We have further introduced an effective Adaptive Fusion Module (AFM) for the dual-path feature aggregation. Extensive experiments on six public deraining benchmarks and downstream vision tasks have demonstrated that our proposed method not only outperforms the existing state-of-the-art deraining method but also achieves visually pleasuring results with excellent robustness on downstream vision tasks.

Dual-Path Coupled Image Deraining Network via Spatial-Frequency Interaction

TL;DR

The paper tackles single-image deraining by integrating spatial and frequency-domain cues through a dual-path architecture named DPCNet. It introduces Spatial Feature Extraction Block (SFEBlock) with a Spatial-Channel Transformer Block (SCTB) to fuse spatial and channel information, Frequency Feature Extraction Block (FFEBlock) based on channel-wise FFT to capture high-frequency textures, and an Adaptive Fusion Module (AFM) to fuse dual-domain features. The approach is implemented in a multi-scale encoder-decoder with Dual-Domain Blocks (DDBlock) and demonstrates state-of-the-art results on six synthetic/real benchmarks, along with improved robustness on downstream object detection tasks. The loss combines , perceptual, and FFT terms, guiding both pixel-level restoration and frequency-domain fidelity. Overall, the work shows the value of jointly leveraging spatial and frequency information for effective deraining and practical vision applications.

Abstract

Transformers have recently emerged as a significant force in the field of image deraining. Existing image deraining methods utilize extensive research on self-attention. Though showcasing impressive results, they tend to neglect critical frequency information, as self-attention is generally less adept at capturing high-frequency details. To overcome this shortcoming, we have developed an innovative Dual-Path Coupled Deraining Network (DPCNet) that integrates information from both spatial and frequency domains through Spatial Feature Extraction Block (SFEBlock) and Frequency Feature Extraction Block (FFEBlock). We have further introduced an effective Adaptive Fusion Module (AFM) for the dual-path feature aggregation. Extensive experiments on six public deraining benchmarks and downstream vision tasks have demonstrated that our proposed method not only outperforms the existing state-of-the-art deraining method but also achieves visually pleasuring results with excellent robustness on downstream vision tasks.
Paper Structure (14 sections, 4 equations, 4 figures, 5 tables)

This paper contains 14 sections, 4 equations, 4 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. PROPOSED METHOD
  4. Spatial Feature Extraction Block
  5. Frequency Feature Extraction Block
  6. Adaptive Fusion Module
  7. Loss Function
  8. Experiments
  9. Implementation Details
  10. Datasets and Compared Methods
  11. Quantitative and Qualitative Experiment
  12. Ablation Studies
  13. Downstream Object Detection Task on Derain Image
  14. Conclusion

Figures (4)

  • Figure 1: Visual comparison on the real datasets, including Real15 JORDER and Real300 Real300. Our method reconstructs credible textures with visually pleasing results.
  • Figure 2: Our proposed DPCNet consists of several Dual-Domain Blocks (DDBlock). The DDBlock is proposed to extract features from both spatial and frequency domains. The Adaptive Fusion Module (AFM) is proposed to interact dimensional information between two branches by reweighting features from the spatial and channel dimensions alternately.
  • Figure 3: Illustrations of our proposed Spatial-Channel Self-Attention (SCSA), which contains spatial-based self-attention and channel-wise self-attention.
  • Figure 4: Visual comparison on the Rain100H JORDER, Rain100L JORDER, Test100 Rain800, Test1200 DIDMDN, and Test2800 DerainNet datasets.