Harnessing Joint Rain-/Detail-aware Representations to Eliminate Intricate Rains

TL;DR

This paper tackles image deraining when training on mixed datasets by learning joint rain-/detail-aware representations that guide adaptive modulation of networks. It introduces CoI-M, a context-based instance-level modulation mechanism, and a rain-/detail-aware contrastive learning strategy, forming the CoIC framework that enhances both CNN and Transformer deraining across diverse rain types and backgrounds. CoIC also provides insights into dataset relationships and enables quantitative assessment of rain and detail contributions to restoration, with substantial improvements on synthetic and real-world data when incorporated into training, including real-world SPAData. The approach achieves superior deraining performance and generalization, demonstrating practical impact for robust outdoor vision systems and offering a path toward all-in-one restoration across varied degradations.

Abstract

Recent advances in image deraining have focused on training powerful models on mixed multiple datasets comprising diverse rain types and backgrounds. However, this approach tends to overlook the inherent differences among rainy images, leading to suboptimal results. To overcome this limitation, we focus on addressing various rainy images by delving into meaningful representations that encapsulate both the rain and background components. Leveraging these representations as instructive guidance, we put forth a Context-based Instance-level Modulation (CoI-M) mechanism adept at efficiently modulating CNN- or Transformer-based models. Furthermore, we devise a rain-/detail-aware contrastive learning strategy to help extract joint rain-/detail-aware representations. By integrating CoI-M with the rain-/detail-aware Contrastive learning, we develop CoIC, an innovative and potent algorithm tailored for training models on mixed datasets. Moreover, CoIC offers insight into modeling relationships of datasets, quantitatively assessing the impact of rain and details on restoration, and unveiling distinct behaviors of models given diverse inputs. Extensive experiments validate the efficacy of CoIC in boosting the deraining ability of CNN and Transformer models. CoIC also enhances the deraining prowess remarkably when real-world dataset is included.

Figures (17)

• Figure 1: (a) rain density distribution. (b) rain-/detail-awareness intensities with respect to rain density. (c) & (d) real-world deraining results of DGUNet mou2022deep trained on three mixed datasets without and with the proposed CoIC, respectively.
• Figure 2: (a) The framework of the proposed CoIC. We extract instance-level representations with the help of rain-/detail-aware contrastive learning strategy. Leveraging these representations as instructive guidance, we then utilize CoI-M to modulate the model's parameters, yielding adaptive deraining results. (b) Generation of rain-/detail-aware negative exemplars.
• Figure 3: The encoder for extracting instance-level representations comprises three components: a feature extractor, GAP layer, and subspace projector. LReLU indicates the LeakyReLU activation.
• Figure 4: Real-world image deraining comparsion on challenging rainy images from RealInt.
• Figure 5: Quantitative comparison of image deraining methods on Rain200H dataset.