Rethinking Real-world Image Deraining via An Unpaired Degradation-Conditioned Diffusion Model

TL;DR

This work addresses real-world image deraining without paired clean/rainy data by introducing RainDiff, an unpaired cycle-consistent framework that couples a degradation-conditioned diffusion model with non-adversarial training. The model learns a latent degradation representation $Z=f_D(\tilde{x})$ and uses a degradation-guided hypernetwork to adapt the reverse diffusion process $p_\theta(x_{t-1}|x_t,\tilde{x})$ across multiple rain types, aided by a multi-scale noise estimator and contrastive learning. Training relies on unpaired data and cycle-consistency losses, avoiding discriminators while enabling deraining across diverse rain degradations. Empirical results on synthetic and real-world datasets show RainDiff outperforms unpaired/semi-supervised methods and approaches fully supervised baselines, highlighting its practical potential for robust real-world deraining and generalization to unseen rain patterns.

Abstract

Recent diffusion models have exhibited great potential in generative modeling tasks. Part of their success can be attributed to the ability of training stable on huge sets of paired synthetic data. However, adapting these models to real-world image deraining remains difficult for two aspects. First, collecting a large-scale paired real-world clean/rainy dataset is unavailable while regular conditional diffusion models heavily rely on paired data for training. Second, real-world rain usually reflects real-world scenarios with a variety of unknown rain degradation types, which poses a significant challenge for the generative modeling process. To meet these challenges, we propose RainDiff, the first real-world image deraining paradigm based on diffusion models, serving as a new standard bar for real-world image deraining. We address the first challenge by introducing a stable and non-adversarial unpaired cycle-consistent architecture that can be trained, end-to-end, with only unpaired data for supervision; and the second challenge by proposing a degradation-conditioned diffusion model that refines the desired output via a diffusive generative process conditioned by learned priors of multiple rain degradations. Extensive experiments confirm the superiority of our RainDiff over existing unpaired/semi-supervised methods and show its competitive advantages over several fully-supervised ones.

Figures (4)

• Figure 1: Image deraining results on a real-world rainy image. From (a) to (f): (a) the real-world rainy image, the deraining results of (b) CycleGAN zhu2017unpaired, (c) DerainCycleGAN wei2021deraincyclegan, (d) DCD-GAN chen2022unpaired, (e) NLCL ye2022unsupervised and (f) our RainDiff. RainDiff generates both rain-free and perceptually more pleasing results.
• Figure 2: The pipeline of RainDiff. It takes unpaired clean/rainy data $\{C,R\}$ as input and trains an unpaired cycle-consistent architecture with a degradation-conditioned diffusion model (DCDM). Once trained, the model can produce high-quality real-world image deraining results, without access to paired clean images. Please refer to Sec. \ref{['proposedmethod']} for details.
• Figure 3: The degradation-conditioned diffusion model.
• Figure 4: Comparison of deraining performance on real-world rainy images. Our method is more successful to remove different rain degradations and obtains the cleanest result with clear details.