NIR-Assisted Image Denoising: A Selective Fusion Approach and A Real-World Benchmark Dataset

TL;DR

The paper tackles low-light RGB image denoising by leveraging paired NIR guidance, addressing content inconsistencies that hinder fusion. It introduces a plug-and-play Selective Fusion Module (SFM) with Global Modulation (GMM) and Local Modulation (LMM) to adaptively fuse NIR-RGB features, plus Real-NAID, a real-world paired RGB-NIR dataset across diverse scenes and noise levels. Empirical results on synthetic and Real-NAID data show state-of-the-art performance gains with lightweight fusion across multiple backbones, and ablations confirm the effectiveness of SFM components and multi-scale loss. This work advances practical NAID by providing both a real-world dataset and a robust fusion module that improves denoising quality in challenging lighting conditions.

Abstract

Despite the significant progress in image denoising, it is still challenging to restore fine-scale details while removing noise, especially in extremely low-light environments. Leveraging near-infrared (NIR) images to assist visible RGB image denoising shows the potential to address this issue, becoming a promising technology. Nonetheless, existing works still struggle with taking advantage of NIR information effectively for real-world image denoising, due to the content inconsistency between NIR-RGB images and the scarcity of real-world paired datasets. To alleviate the problem, we propose an efficient Selective Fusion Module (SFM), which can be plug-and-played into the advanced denoising networks to merge the deep NIR-RGB features. Specifically, we sequentially perform the global and local modulation for NIR and RGB features, and then integrate the two modulated features. Furthermore, we present a Real-world NIR-Assisted Image Denoising (Real-NAID) dataset, which covers diverse scenarios as well as various noise levels. Extensive experiments on both synthetic and our real-world datasets demonstrate that the proposed method achieves better results than state-of-the-art ones. The dataset, codes, and pre-trained models will be publicly available at https://github.com/ronjonxu/NAID.

Figures (12)

• Figure 1: Examples of the structure discrepancy between NIR-RGB images. ($\mathbf{a}$) The structure is visible in the RGB image but not in the NIR image, as shown in the red box. ($\mathbf{b}$) The structure is visible in the NIR image but not in the RGB image, as shown in the yellow box.
• Figure 2: The construction of Real-NAID dataset. ($\mathbf{a}$) Capture noisy RGB images with three types of high ISO and short exposure time. ($\mathbf{b}$) Capture clean RGB images with low ISO and long exposure time. ($\mathbf{c}$) Turn on the NIR light, then capture the clean NIR images with low ISO and short exposure time.
• Figure 3: Comparison of different image denoising manners using multi-scale architecture. ($\mathbf{a}$) RGB image denoising. ($\mathbf{b}$) NIR-assisted RGB image denoising baseline. ($\mathbf{c}$) NIR-assisted RGB image denoising with our proposed Selective Fusion Module (SFM).
• Figure 4: The structure of our proposed Selective Fusion Module (SFM), where Global Modulation Module (GMM) and Local Modulation Module (LMM) focus on color and structure discrepancy issues between the NIR images and RGB ones, respectively. Two $1 \times 1$ blocks and $5 \times 5$ blocks are used in GMM and LMM, respectively.
• Figure 5: Qualitative comparison on synthetic DVD dataset. Bold marks our methods.