Exploring the Potentials of Spiking Neural Networks for Image Deraining

TL;DR

This work investigates whether Spiking Neural Networks can excel at a low-level vision task, image deraining, by revealing that traditional LIF neurons act as intensity-triggered, high-frequency indicators. It introduces Visual LIF (VLIF) to incorporate local visual context and stabilizes learning with NI-LIF, plus Spiking Decomposition & Enhancement Module (SDEM) and Spiking Multi-scale Unit (SMU) to enable hierarchical, multi-scale feature refinement. Across five datasets, the approach outperforms prior SNN deraining methods and remains competitive with CNN/Transformer baselines while consuming significantly less energy (about 13% of competing SNN methods). The results also demonstrate task-adaptive behavior in a related UIE setting, suggesting broad promise for energy-efficient SNNs in dense image restoration tasks.

Abstract

Biologically plausible and energy-efficient frameworks such as Spiking Neural Networks (SNNs) have not been sufficiently explored in low-level vision tasks. Taking image deraining as an example, this study addresses the representation of the inherent high-pass characteristics of spiking neurons, specifically in image deraining and innovatively proposes the Visual LIF (VLIF) neuron, overcoming the obstacle of lacking spatial contextual understanding present in traditional spiking neurons. To tackle the limitation of frequency-domain saturation inherent in conventional spiking neurons, we leverage the proposed VLIF to introduce the Spiking Decomposition and Enhancement Module and the lightweight Spiking Multi-scale Unit for hierarchical multi-scale representation learning. Extensive experiments across five benchmark deraining datasets demonstrate that our approach significantly outperforms state-of-the-art SNN-based deraining methods, achieving this superior performance with only 13\% of their energy consumption. These findings establish a solid foundation for deploying SNNs in high-performance, energy-efficient low-level vision tasks.

Figures (6)

• Figure 1: LIF $f(\cdot)$ highlights high-frequency rain but exhibits frequency saturation with repeated applications after $t$=$1$. $x$ is embedded features, $fre$ is frequency spectrums.
• Figure 2: The comparison of LIF and VLIF. (a) shows differences in pipeline, (b) shows differences response areas, in the top-right insets, blue and orange denote the response for LIF and VLIF, respectively.
• Figure 3: Activation comparison. VLIF activates $3.58\times$ more than LIF by incorporating local spatial context, enabling better feature representation in low-response regions.
• Figure 4: (a) Architecture; (b) Spiking Decomposition & Enhancement Module (SDEM), (c) Spiking Multi-scale Unit (SMU).
• Figure 5: Qualitative comparison on the Rain200H. Our model generates finer-grained details and clearer textures.