Image Deraining via Self-supervised Reinforcement Learning

TL;DR

This work tackles outdoor rain removal by introducing SRL-Derain, a self-supervised reinforcement-learning framework for image deraining. The method locates rain streaks via a Rain Dictionary Prior and uses a multi-agent pixel-wise RL scheme (A3C) to progressively inpaint rain regions, guided by self-supervised rewards from pseudo-derained references and a no-reference quality metric BRISQUE. It is presented as the first SSL-RL approach to deraining, and experiments show SRL-Derain outperforms state-of-the-art few-shot and self-supervised methods on multiple benchmarks, with competitive real-world performance. The work demonstrates the practicality of combining dictionary-based rain detection with SSL-driven RL for robust deraining without paired training data, suggesting broader applicability of SSL-RL to low-level vision tasks.

Abstract

The quality of images captured outdoors is often affected by the weather. One factor that interferes with sight is rain, which can obstruct the view of observers and computer vision applications that rely on those images. The work aims to recover rain images by removing rain streaks via Self-supervised Reinforcement Learning (RL) for image deraining (SRL-Derain). We locate rain streak pixels from the input rain image via dictionary learning and use pixel-wise RL agents to take multiple inpainting actions to remove rain progressively. To our knowledge, this work is the first attempt where self-supervised RL is applied to image deraining. Experimental results on several benchmark image-deraining datasets show that the proposed SRL-Derain performs favorably against state-of-the-art few-shot and self-supervised deraining and denoising methods.

Figures (6)

• Figure 1: A comparison of deraining results between R2A peng2023rain2avoid, a self-supervised learning method, and SRL-Derain, the proposed self-supervised reinforcement-learning method.
• Figure 2: The flowchart of the proposed self-supervised RL-based deraining scheme. To locate rain pixels and generate the rain mask, we utilize bilateral filtering to extract the high-frequency part $I_{HF}$ from the input rain image and decompose the rain components via dictionary learning decompositionSID_Kang2011TIP. The RL model progressively fills the rain pixels in the rain image based on the rain mask. Note that $r^{t} \in R^{H\times W}$ is the total reward map for the state $s^t$ at the time step $t$.
• Figure 3: Qualitative comparisons on Rain100L with PSNR/SSIM values shown on the results. (a) Input images, and the derained results obtained using (b) DIP ulyanov2018deep, (c) N2S batson2019noise2self, (d) N2V krull2019noise2void, (e) R2A peng2023rain2avoid, and (f) Ours. (g) GT images.
• Figure 4: Qualitative comparisons on Rain800 with PSNR/SSIM values shown on the results. (a) Input images, and the derained results obtained using (b) DIP ulyanov2018deep, (c) N2S batson2019noise2self, (d) N2V krull2019noise2void, (e) R2A peng2023rain2avoid, and (f) Ours. (g) GT images.
• Figure 5: Qualitative comparisons on DDN-SIRR_syn with PSNR/SSIM values shown on the results. (a) Input images, and the derained results obtained using (b) DIP ulyanov2018deep, (c) N2S batson2019noise2self, (d) N2V krull2019noise2void, (e) R2A peng2023rain2avoid, and (f) Ours. (g) GT images.