Deep Joint Unrolling for Deblurring and Low-Light Image Enhancement (JUDE)

TL;DR

JUDE tackles the joint problem of low-light image enhancement and deblurring by grounding a deep unrolled network in a Retinex-based physical model. It formulates a joint optimization with deblurring and illumination-reflectance decomposition terms, solved via ALM and implemented as a multi-block unrolled architecture that uses CNN priors for certain variables. The method integrates a Kernel Estimation module, Illuminance Enhancement, and Reflectance Denoising to produce sharp, well-lit outputs, and trains end-to-end on synthetic LOL-Blur data while validating on Real-LOL-Blur. Quantitatively and qualitatively, JUDE outperforms specialized baselines on both synthetic and real night scenes, demonstrating strong generalization and interpretability due to its model-based roots.

Abstract

Low-light and blurring issues are prevalent when capturing photos at night, often due to the use of long exposure to address dim environments. Addressing these joint problems can be challenging and error-prone if an end-to-end model is trained without incorporating an appropriate physical model. In this paper, we introduce JUDE, a Deep Joint Unrolling for Deblurring and Low-Light Image Enhancement, inspired by the image physical model. Based on Retinex theory and the blurring model, the low-light blurry input is iteratively deblurred and decomposed, producing sharp low-light reflectance and illuminance through an unrolling mechanism. Additionally, we incorporate various modules to estimate the initial blur kernel, enhance brightness, and eliminate noise in the final image. Comprehensive experiments on LOL-Blur and Real-LOL-Blur demonstrate that our method outperforms existing techniques both quantitatively and qualitatively.

Figures (4)

• Figure 1: JUDE Architecture. First, all variables and multipliers are initialized with zeros. The input $\mathbfcal{X}$ is put through a kernel estimation module, and outputs an initial blur kernel $\mathbfcal{H}$ and a weight map. The proposed JUDE consists of K blocks, wherein each block, $\mathbfcal{I}$, $\mathbfcal{R}$, and $\mathbfcal{L}$ are updated using closed-form solutions, whereas $\mathbfcal{Z}$, $\mathbfcal{P}$ and $\mathbfcal{Q}$ are updated utilizing Convolutional Neural Networks (CNNs), particularly a ResUNet. Decomposed illuminance $\mathbfcal{L}$ is enhanced, and reflectance $\mathbfcal{R}$ is denoised before merging into a non-blurry, bright output.
• Figure 2: The architecture of the Kernel Estimation (left), Illuminance Enhancement (right), and Reflectance Denoiser (middle) modules.
• Figure 3: Visual comparison on LOL-Blur dataset. The yellow box indicates obvious differences and is shown at the bottom-left of each result image. (Zoom in for better visualization.)
• Figure 4: Visual comparison on Real-LOL-Blur rim2020real dataset. The yellow box indicates obvious differences and is shown at the bottom-left of each result image. (Zoom in for better visualization.)