Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Gabor-guided transformer for single image deraining

Sijin He, Guangfeng Lin

TL;DR

Gabformer presents a Gabor-guided transformer for single-image deraining that injects multi-scale texture information into the attention mechanism to preserve high-frequency details. It uses cross-channel MGSA with Q derived from Gabor-filtered features and K,V from channel-context, complemented by a gating FFN to filter uninformative high-frequency content. Empirical results on Rain200L/H, Rain200H, DID-Data, DDN-Data, and AGAN-Data show state-of-the-art PSNR/SSIM and faithful texture restoration, with a model size of about 34.4M parameters. The work demonstrates the value of integrating traditional texture filters with transformer-based restoration and points to future work in model compression for practical deployment.

Abstract

Image deraining have have gained a great deal of attention in order to address the challenges posed by the effects of harsh weather conditions on visual tasks. While convolutional neural networks (CNNs) are popular, their limitations in capturing global information may result in ineffective rain removal. Transformer-based methods with self-attention mechanisms have improved, but they tend to distort high-frequency details that are crucial for image fidelity. To solve this problem, we propose the Gabor-guided tranformer (Gabformer) for single image deraining. The focus on local texture features is enhanced by incorporating the information processed by the Gabor filter into the query vector, which also improves the robustness of the model to noise due to the properties of the filter. Extensive experiments on the benchmarks demonstrate that our method outperforms state-of-the-art approaches.

Gabor-guided transformer for single image deraining

TL;DR

Gabformer presents a Gabor-guided transformer for single-image deraining that injects multi-scale texture information into the attention mechanism to preserve high-frequency details. It uses cross-channel MGSA with Q derived from Gabor-filtered features and K,V from channel-context, complemented by a gating FFN to filter uninformative high-frequency content. Empirical results on Rain200L/H, Rain200H, DID-Data, DDN-Data, and AGAN-Data show state-of-the-art PSNR/SSIM and faithful texture restoration, with a model size of about 34.4M parameters. The work demonstrates the value of integrating traditional texture filters with transformer-based restoration and points to future work in model compression for practical deployment.

Abstract

Image deraining have have gained a great deal of attention in order to address the challenges posed by the effects of harsh weather conditions on visual tasks. While convolutional neural networks (CNNs) are popular, their limitations in capturing global information may result in ineffective rain removal. Transformer-based methods with self-attention mechanisms have improved, but they tend to distort high-frequency details that are crucial for image fidelity. To solve this problem, we propose the Gabor-guided tranformer (Gabformer) for single image deraining. The focus on local texture features is enhanced by incorporating the information processed by the Gabor filter into the query vector, which also improves the robustness of the model to noise due to the properties of the filter. Extensive experiments on the benchmarks demonstrate that our method outperforms state-of-the-art approaches.
Paper Structure (10 sections, 6 equations, 4 figures, 4 tables)

This paper contains 10 sections, 6 equations, 4 figures, 4 tables.

Table of Contents

  1. Introduction
  2. PROPOSED METHOD
  3. Overall pipeline
  4. Multi-Gabor Self Attention
  5. Gating Feed-Forward Network
  6. EXPERIMENTAL RESULTS
  7. Experimental Settings
  8. Ablation Study
  9. Experimental Analysis
  10. CONCLUSIONS

Figures (4)

  • Figure 1: Visual comparison effect of our method with IDT2022IDT. HF is the visualization result after high-pass filtering. Unlike the IDT, which adds priors to the network structure, our method extract the texture through Gabor, which can recover a finer effect.
  • Figure 2: Detailed framework of Gabformer with the main constituent modules of (a) overall framework(Gabformer), (b) Multi-Gabor Self Attention (MGSA), (c) Gabor Filter (GAB), (d) Gated Feed-Forward Network (GFFN)
  • Figure 3: Comparison of rain removal visualization on the Rain200L dataset, zoom in for clearer view of effectiveness.
  • Figure 4: Comparison of rain removal visualization on the AGAN-Data dataset, zoom in for clearer view of effectiveness.