Cross-Scenario Deraining Adaptation with Unpaired Data: Superpixel Structural Priors and Multi-Stage Pseudo-Rain Synthesis

Abstract

Image deraining plays a pivotal role in low-level computer vision, serving as a prerequisite for robust outdoor surveillance and autonomous driving systems. While deep learning paradigms have achieved remarkable success in firmly aligned settings, they often suffer from severe performance degradation when generalized to unseen Out-of-Distribution (OOD) scenarios. This failure stems primarily from the significant domain discrepancy between synthetic training datasets and the complex physical dynamics of real-world rain. To address these challenges, this paper proposes a pioneering cross-scenario deraining adaptation framework. Diverging from conventional approaches, our method obviates the requirements for paired rainy observations in the target domain, leveraging exclusively rain-free background images. We design a Superpixel Generation (Sup-Gen) module to extract stable structural priors from the source domain using Simple Linear Iterative Clustering. Subsequently, a Resolution-adaptive Fusion strategy is introduced to align these source structures with target backgrounds through texture similarity, ensuring the synthesis of diverse and realistic pseudo-data. Finally, we implement a pseudo-label re-Synthesize mechanism that employs multi-stage noise generation to simulate realistic rain streaks. This framework functions as a versatile plug-and-play module capable of seamless integration into arbitrary deraining architectures. Extensive experiments on state-of-the-art models demonstrate that our approach yields remarkable PSNR gains of up to 32% to 59% in OOD domains while significantly accelerating training convergence.

Figures (7)

• Figure 1: The challenge of removing raindrop across various Out-of-distribution (OOD) scenarios. We first use the Rain200L dataset to train the NeRD-Rain, DRSformer, and FADformer methods respectively. In order to implement the evaluation on Out-of-distribution(OOD) scenarios, we use Rain200L as the source domain data, and Rain200H, DID, and DDN as the OOD domains for testing. Empirical results demonstrate that while the model achieves exemplary deraining performance on the source domain (Rain200L), the performance of state-of-the-art derain models deteriorates significantly when applied to unseen OOD domains. Thus, we call for a solid derain pipeline is required to better handle unknown scene conditions.
• Figure 2: An illustration of our proposed method: (A) Superpixel Generation: This module takes the source domain rain-free image $I_{source}$ as input. Utilizing a superpixel segmentation algorithm, it parses the image into a collection of superpixel blocks rich in structural and textural information, denoted as $I_{target}^{superpixel}$. (B) Resolution-adaptive Fusion: This module performs local matching between the extracted superpixel blocks $I_{target}^{superpixel}$ and the target domain’s pseudo rain-free image $I_{target}^{psdo}$. It locates optimal integration regions $\mathbf{P}_p^*$ based on semantic consistency and texture similarity(MSE). By employing a random mask matrix $M_{\text{random}}$ for proportional extraction and applying an $\alpha$-fusion strategy with a fusion coefficient of $\beta$, it generates an information-enhanced target image, $I_{target}^{fuse}$. (C) Pseudo-label Re-Synthesis: To synthesize high-fidelity pseudo rain streak layers, the module implements a three-stage degradation process comprising Salt-and-Pepper noise ($X_s$), Gaussian blurring ($X_g$), and motion blurring ($X_m$). This streak layer is superimposed onto the luminance channel of $I_{target}^{fuse}$ using $\alpha$ fusion to create the corresponding rainy image, $I_{input}^{fuse}$. With the synergistic operation of these three modules, a final set of pseudo-paired samples $(I_{input}^{fuse}, I_{target}^{fuse})$ is obtained, significantly enhancing the model's generalization toward out-of-distribution (OOD) domains.
• Figure 3: The framework of Superpixel Generation, the process initiates by incorporating spatial positional information $(x,y)$ into the image data $(l,a,b)$. Subsequently, cluster centroids $(l_i,a_i,b_i,x_i,y_i)$ are initialized based on predefined parameters.
• Figure 4: The framework of Resolution-Adaptive Fusion. We propose a novel fusion paradigm driven by bidirectional region matching and adaptive fusion mechanisms. This approach simultaneously incorporates structural priors from the source domain while preserving the intrinsic background distribution of the target domain, thereby facilitating subsequent domain adaptation tasks.
• Figure 5: Pseudo-label Re-Synthesis introduces a pseudo-rain streak generation methodology grounded in multi-stage noise synthesis. By sequentially executing noise initialization, morphological transformation, and luminance channel fusion within the YUV color space, the proposed method achieves the realistic simulation of rain effects on rain-free images within the target domain.