Dual Degradation Representation for Joint Deraining and Low-Light Enhancement in the Dark

TL;DR

This work tackles the challenging problem of restoring images captured under simultaneous nighttime illumination and rain. It introduces L$^{2}$RIRNet, an end-to-end framework consisting of a Dual Degradation Representation Network (DDR-Net) and a Restoration Net, with a Fourier Detail Guidance (FDG) module and a dual degradation loss (DDLoss) to disentangle and utilize bright-region rain and dark-region low-light degradations. The method learns degradation representations for bright and dark regions separately and leverages near-rainless detail information to guide restoration, achieving superior performance on a newly created Low-Light-Rainy (LLR) dataset in both synthetic and real-world scenarios. The approach yields significant improvements over cascaded and retrained baselines, demonstrating practical impact for night-time autonomous driving, surveillance, and photography by delivering cleaner, better-exposed images with reduced rain artifacts.

Abstract

Rain in the dark poses a significant challenge to deploying real-world applications such as autonomous driving, surveillance systems, and night photography. Existing low-light enhancement or deraining methods struggle to brighten low-light conditions and remove rain simultaneously. Additionally, cascade approaches like ``deraining followed by low-light enhancement'' or the reverse often result in problematic rain patterns or overly blurred and overexposed images. To address these challenges, we introduce an end-to-end model called L$^{2}$RIRNet, designed to manage both low-light enhancement and deraining in real-world settings. Our model features two main components: a Dual Degradation Representation Network (DDR-Net) and a Restoration Network. The DDR-Net independently learns degradation representations for luminance effects in dark areas and rain patterns in light areas, employing dual degradation loss to guide the training process. The Restoration Network restores the degraded image using a Fourier Detail Guidance (FDG) module, which leverages near-rainless detailed images, focusing on texture details in frequency and spatial domains to inform the restoration process. Furthermore, we contribute a dataset containing both synthetic and real-world low-light-rainy images. Extensive experiments demonstrate that our L$^{2}$RIRNet performs favorably against existing methods in both synthetic and complex real-world scenarios. All the code and dataset can be found in \url{https://github.com/linxin0/Low_light_rainy}.

Figures (12)

• Figure 1: Motivation. Results from existing low-light enhancement and deraining methods on synthetic (1st and 2nd rows) and real-world (3rd and 4th rows) low-light-rainy images. (a) Input; (b) DRS drsformer; (c) PairLIE pair; (d) DRS drsformer + PairLIE pair; (e) PairLIE pair + DRS drsformer; (f) L$^{2}$RIRNet (Ours). While single-task models can only address either low-light enhancement or deraining, the cascaded methods may result in issues such as overexposure, underexposure, residual rain patterns, and blurring. Our L$^{2}$RIRNet can achieve better results. The results highlight the challenges of existing approaches in effectively addressing low light and rain conditions.
• Figure 2: Overview of our L$^{2}$RIRNet consists of two components: DDR-Net and Restoration-Net. DDR-Net extracts the image's degradation representation of bright and dark regions by rainy and light maps, respectively. The dual degradation loss (DDLoss) is presented to constrain the training process. The decoder generates the multi-channel latent features to guide the restoration process effectively. The Fourier Detailed Guidance (FDG) module uses prior near-rainless information for the Restoration-Net. The training process consists of two phases: 1. Train both DDR-Net and Restoration-Net to get effective representations. 2. Fix the DDR-Net and only train the Restoration-Net to enhance restoration performance further.
• Figure 3: The illustration of the relationship between rain pattern density (m) and the Euclidean distance (r) from the pixel to the light source.
• Figure 4: (a) DASR's patch-based positive and negative label approach. The selected patches may come from low-light and rain pattern areas, which is challenging to apply as a positive label. (b) Our image-based method learns degradation representation by the dual degradation loss. We present the rainy and light encoder part of the DDR-Net, which takes in three images: a low-light-rainy image (Input), an augmented version of that image ($Input_{aug}$), and a paired clean image (Target). These images are encoded using a parameter-sharing network, with rainy and light maps focusing on the degradation representation in bright and dark regions. A multilayer perceptron (MLP) then processes the resulting feature information to obtain the degradation operator for comparison learning constraint. The $DD_{Image}$ is the dual degradation representation from Input, the $DD_{+}$ is the dual degradation representation from $Input_{aug}$, and the $DD_{-}$ is the dual degradation representation from Target. Our approach aims to make $DD_{Image}$ and $DD_{+}$ similar while keeping $DD$ and $DD_{-}$ distinct.
• Figure 5: Visual presentation of detailed images. Compared to the input, the detailed image removes most raindrops while maintaining the content information.