Channel Consistency Prior and Self-Reconstruction Strategy Based Unsupervised Image Deraining

TL;DR

This work tackles the real-world challenge of image deraining without access to paired clean-rainy data by introducing CSUD, an unsupervised framework that generates high-quality pseudo paired data through a rain-aware generator guided by real rain, a derainer, and a PatchGAN discriminator. Two key innovations, Channel Consistency Loss (CCLoss) based on the Channel Consistency Prior (CCP) and a Self-Reconstruction (SR) strategy, constrain the generator to preserve background details while accurately transferring rain streaks and to minimize redundant information transfer. Extensive experiments on synthetic and real-world datasets show that CSUD outperforms existing unsupervised methods and exhibits strong generalization, often approaching or matching supervised methods on real rain scenarios. The approach advances unsupervised deraining with practical impact on deployment in real-world systems by reducing the dependence on paired data and enhancing cross-domain robustness.

Abstract

Recently, deep image deraining models based on paired datasets have made a series of remarkable progress. However, they cannot be well applied in real-world applications due to the difficulty of obtaining real paired datasets and the poor generalization performance. In this paper, we propose a novel Channel Consistency Prior and Self-Reconstruction Strategy Based Unsupervised Image Deraining framework, CSUD, to tackle the aforementioned challenges. During training with unpaired data, CSUD is capable of generating high-quality pseudo clean and rainy image pairs which are used to enhance the performance of deraining network. Specifically, to preserve more image background details while transferring rain streaks from rainy images to the unpaired clean images, we propose a novel Channel Consistency Loss (CCLoss) by introducing the Channel Consistency Prior (CCP) of rain streaks into training process, thereby ensuring that the generated pseudo rainy images closely resemble the real ones. Furthermore, we propose a novel Self-Reconstruction (SR) strategy to alleviate the redundant information transfer problem of the generator, further improving the deraining performance and the generalization capability of our method. Extensive experiments on multiple synthetic and real-world datasets demonstrate that the deraining performance of CSUD surpasses other state-of-the-art unsupervised methods and CSUD exhibits superior generalization capability.

Figures (21)

• Figure 1: Deraining results on the real rainy images captured by ourselves in real-world scenarios. Compared with the supervised method Restormer restormer and the unsupervised method DerainCycleGAN DerainCycleGAN, our CSUD exhibits extremely strong generalization capability in the real world and achieves the best visual results.
• Figure 1: Deraining results on the real rainy images captured by ourselves in real-world scenarios. Compared with the supervised method Restormer restormer and the unsupervised method DerainCycleGAN DerainCycleGAN, our CSUD exhibits extremely strong generalization capability and achieves the best visual results.
• Figure 2: Framework of the proposed CSUD. The red arrows represent the baseline, purple arrows represent the 3 extra constraints, green arrows represent the self-reconstruction (SR) strategy, and blue arrows represent the CCLoss. The 4 input arrows of RIEM represent 4 separate runs of the generator, each time guided by a different image. Both RIEM and CFEM receive only 1 input image at a time.
• Figure 2: Visualization of channel consistency prior in rainy images. From top to bottom, the 3 sets of images are synthetic, real-world, and real-world nighttime images, respectively. The first column presents the clean and rainy RGB images; the second, third and fourth columns present their R, G, B channels, respectively; the fifth, sixth, and seventh columns present the cycle subtractions of R, G, and B channels of clean and rainy images, respectively.
• Figure 3: (a) Visualization of channel consistency prior in synthetic and real-world rainy images. The first column presents the rainy RGB images; the second, third and fourth columns present their R, G, B channels, respectively; the fifth, sixth, and seventh columns present the cycle subtractions of R, G, and B channels, respectively. (b) Cosine similarities of the cycle subtractions of R, G, and B channels between clean and rainy images in the 5 synthetic and real-world datasets. More visualizations and analysis are presented in the Appendix.