Reti-Diff: Illumination Degradation Image Restoration with Retinex-based Latent Diffusion Model

TL;DR

Reti-Diff tackles illumination-degraded image restoration by embedding diffusion in a compact latent space and guiding restoration with Retinex priors. It introduces RLDM to learn reflectance and illumination priors from low-quality inputs and RGformer to decompose and refine features under Retinex guidance. A two-phase training scheme first learns priors from ground-truth data and then trains RLDM to predict consistent priors from low-quality inputs, achieving state-of-the-art results on IDIR tasks and favorable downstream performance. The approach delivers substantial efficiency gains and robust performance across challenging degradation scenarios, indicating strong potential for real-world imaging and vision tasks.

Abstract

Illumination degradation image restoration (IDIR) techniques aim to improve the visibility of degraded images and mitigate the adverse effects of deteriorated illumination. Among these algorithms, diffusion model (DM)-based methods have shown promising performance but are often burdened by heavy computational demands and pixel misalignment issues when predicting the image-level distribution. To tackle these problems, we propose to leverage DM within a compact latent space to generate concise guidance priors and introduce a novel solution called Reti-Diff for the IDIR task. Reti-Diff comprises two key components: the Retinex-based latent DM (RLDM) and the Retinex-guided transformer (RGformer). To ensure detailed reconstruction and illumination correction, RLDM is empowered to acquire Retinex knowledge and extract reflectance and illumination priors. These priors are subsequently utilized by RGformer to guide the decomposition of image features into their respective reflectance and illumination components. Following this, RGformer further enhances and consolidates the decomposed features, resulting in the production of refined images with consistent content and robustness to handle complex degradation scenarios. Extensive experiments show that Reti-Diff outperforms existing methods on three IDIR tasks, as well as downstream applications. Code will be available at \url{https://github.com/ChunmingHe/Reti-Diff}.

Figures (11)

• Figure 1: Our Reti-Diff outperforms cutting-edge techniques on three IDIR tasks and the low-light object detection task, where CLIP, Diff-Re, and SNR are short for CLIP-LIT liang2023iterative, Diff-Retinex yi2023diff, and SNR-Net xu2022snr.
• Figure 2: Framework of Reti-Diff. We first pretrain Reti-Diff to ensure the robust learning of RLDM and then optimize RLDM to generate high-quality Retinex priors, which guide RGformer in detail enhancement and illumination correction. In (a), we omit the auxiliary decoder $D_a(\mathbin{\vcenter{\hbox{$\m@th\bullet$}}})$ for simplicity. In (c), we only give the example by using RLDM to extract the reflectance prior and the illumination prior can be extracted similarly.
• Figure 3: Visual results on the low-light image enhancement task.
• Figure 4: Visual results on the underwater image enhancement task.
• Figure 5: Visual results on the backlit image enhancement task.