Rethinking Nighttime Image Deraining via Learnable Color Space Transformation

TL;DR

The paper tackles nighttime image deraining by acknowledging illumination-driven nonuniform rain distribution and limited realistic datasets. It introduces HQ-NightRain, a high-quality nighttime rain benchmark synthesized with an illumination-aware pipeline, and CST-Net, a two-stage framework that performs rain removal in the luminance Y channel using a learnable CSC and an Implicit Illumination Guidance module. The authors demonstrate state-of-the-art performance across HQ-NightRain and public nighttime benchmarks, with ablations confirming the effectiveness of the learnable color space conversion, the Y-channel focus, and the illumination-guided design. The work yields practical impact by improving deraining quality and downstream vision tasks under challenging nighttime conditions, and it provides a versatile data synthesis approach for realistic nighttime rain in applications like autonomous driving and film/game production.

Abstract

Compared to daytime image deraining, nighttime image deraining poses significant challenges due to inherent complexities of nighttime scenarios and the lack of high-quality datasets that accurately represent the coupling effect between rain and illumination. In this paper, we rethink the task of nighttime image deraining and contribute a new high-quality benchmark, HQ-NightRain, which offers higher harmony and realism compared to existing datasets. In addition, we develop an effective Color Space Transformation Network (CST-Net) for better removing complex rain from nighttime scenes. Specifically, we propose a learnable color space converter (CSC) to better facilitate rain removal in the Y channel, as nighttime rain is more pronounced in the Y channel compared to the RGB color space. To capture illumination information for guiding nighttime deraining, implicit illumination guidance is introduced enabling the learned features to improve the model's robustness in complex scenarios. Extensive experiments show the value of our dataset and the effectiveness of our method. The source code and datasets are available at https://github.com/guanqiyuan/CST-Net.

Figures (28)

• Figure 1: Histograms of different color channels show that the Y channel in the YCbCr color space demonstrates the most significant difference between rainy and rain-free images at nighttime.
• Figure 2: Data construction pipeline of previous method and our proposed approach. Existing methods add global rain effects linearly onto nighttime backgrounds to generate rainy images. Insetad, our approach takes illumination information in nighttime scenes into account to synthesize nighttime rainy images with higher harmony and realism.
• Figure 3: (a) Sample images from different datasets. Our dataset carefully considers harmony in synthesizing nighttime rainy scenes, resulting in more visually realistic appearances. (b) To avoid the influence of background content on the results, we select samples with similar nighttime street backgrounds and use ResNet50 to extract features. Images from the same dataset exhibit clustering after t-SNE dimensionality reduction due to the similarity of rain streak features. The small domain gap indicates that our dataset synthesis pipeline generates nighttime rain that is closer to reality.
• Figure 4: Overall architecture of the proposed end-to-end CST-Net for nighttime image deraining, which consists of a degradation removal stage and a color refinement stage. Among these two stages, we develop a color space converter (CSC) to achieve space transformation (RGB $\leftrightarrows$ YCbCr), and also construct an implicit illumination guidance (IIG) branch to better transmit illumination information.
• Figure 5: Derained results on the HQ-NightRain (first row) and GTAV-NightRain zhang2022gtav (second row).