Unsupervised Network for Single Image Raindrop Removal

TL;DR

The paper tackles single-image raindrop removal without paired data by proposing an unsupervised image-decomposition framework that splits a rainy image into a clean background $B$, a raindrop layer $R$, and a transparency mask $\alpha$, modeled as $I = (1 - \alpha) \odot B + \alpha \odot R$. A cycle-consistent adversarial framework with two discriminators and a single generator enforces plausible decomposition and reconstruction, while an iterative neural network with a feedback mechanism refines the decomposition over $N$ iterations. The loss combines $L_{GAN}$, $L_{cyc}$, $L_{identity}$, and $L_{sparsity}$, with iteration-aware weighting, yielding notable PSNR/SSIM gains on the NUS Raindrop and RainDS datasets and demonstrating robustness to varied raindrop shapes and densities. This approach advances practical raindrop removal by eliminating the need for paired data and leveraging progressive, top-down refinement for accurate layer separation.

Abstract

Image quality degradation caused by raindrops is one of the most important but challenging problems that reduce the performance of vision systems. Most existing raindrop removal algorithms are based on a supervised learning method using pairwise images, which are hard to obtain in real-world applications. This study proposes a deep neural network for raindrop removal based on unsupervised learning, which only requires two unpaired image sets with and without raindrops. Our proposed model performs layer separation based on cycle network architecture, which aims to separate a rainy image into a raindrop layer, a transparency mask, and a clean background layer. The clean background layer is the target raindrop removal result, while the transparency mask indicates the spatial locations of the raindrops. In addition, the proposed model applies a feedback mechanism to benefit layer separation by refining low-level representation with high-level information. i.e., the output of the previous iteration is used as input for the next iteration, together with the input image with raindrops. As a result, raindrops could be gradually removed through this feedback manner. Extensive experiments on raindrop benchmark datasets demonstrate the effectiveness of the proposed method on quantitative metrics and visual quality.

Figures (9)

• Figure 1: Examples of rainy images in the datasets we used. (a) and (b) are from the NUS raindrop dataset qian2018attentive. (c) and (d) are from RainDS Quan2021InOneGo.
• Figure 2: Overall flowchart of our proposed method.
• Figure 3: Diagram of Image Decomposition. A heat map represents the transparency mask. The closer the value of a pixel is to 1, the closer its color is to red.
• Figure 4: Detailed flowchart of the iterative network as the generator in Fig. \ref{['fig31']}.
• Figure 5: Samples of rainy images, transparency mask, and clean background images at different iterations in our iterative neural network.