Complementary Advantages: Exploiting Cross-Field Frequency Correlation for NIR-Assisted Image Denoising

TL;DR

This work tackles the problem of denoising RGB images by leveraging complementary information from co-registered NIR images. By uncovering a cross-field frequency correlation prior, the authors design FCENet, a two-stage, frequency-domain fusion network with a Frequency Dynamic Selection Mechanism (FDSM) and a Frequency Exhaustive Fusion Mechanism (FEFM) to selectively and thoroughly fuse NIR and RGB features. The CFR and DFR components within FEFM jointly reinforce common frequency patterns while preserving differential high-frequency cues, guided by a frequency-domain loss term. Experiments on the DVD and IVRG datasets, along with real-world data, show that FCENet outperforms state-of-the-art methods in PSNR/SSIM and detail preservation, while remaining efficient; the authors also provide ablations that validate the contributions of FDSM and FEFM.

Abstract

Existing single-image denoising algorithms often struggle to restore details when dealing with complex noisy images. The introduction of near-infrared (NIR) images offers new possibilities for RGB image denoising. However, due to the inconsistency between NIR and RGB images, the existing works still struggle to balance the contributions of two fields in the process of image fusion. In response to this, in this paper, we develop a cross-field Frequency Correlation Exploiting Network (FCENet) for NIR-assisted image denoising. We first propose the frequency correlation prior based on an in-depth statistical frequency analysis of NIR-RGB image pairs. The prior reveals the complementary correlation of NIR and RGB images in the frequency domain. Leveraging frequency correlation prior, we then establish a frequency learning framework composed of Frequency Dynamic Selection Mechanism (FDSM) and Frequency Exhaustive Fusion Mechanism (FEFM). FDSM dynamically selects complementary information from NIR and RGB images in the frequency domain, and FEFM strengthens the control of common and differential features during the fusion process of NIR and RGB features. Extensive experiments on simulated and real data validate that the proposed method outperforms other state-of-the-art methods. The code will be released at https://github.com/yuchenwang815/FCENet.

Figures (9)

• Figure 1: Comparisons of PSNR, FLOPs, and Parameters on the DVD dataset jin2022darkvisionnet are presented. Green circles represent the single-image denoising algorithms, and blue circles represent the NIR-assisted image denoising algorithms. The circle radius represents the number of parameters. The proposed method (red circles) achieves the superior performance while maintaining efficiency.
• Figure 2: Visual comparisons on the challenging noisy RGB-NIR image pairs. Pay attention to the differences in structural and color information between two fields. Our method produces a better denoising result with clear details and fewer artifacts.
• Figure 3: The analysis of NIR and RGB frequency correlation. (a) The frequency correlation between noisy RGB and clean RGB, with the horizontal axis indicating the cutoff frequency of the high-pass filter. Different colored curves represent different scenarios. (b) The frequency correlation between NIR and clean RGB. (c) The visualization results of images output by a fixed frequency filter, with the first row showing high frequency (HF) part and the second row showing low frequency (LF) part.
• Figure 4: The overall architecture of the proposed cross-field Frequency Correlation Exploiting Network (FCENet). In the first stage, pre-denoising is performed. In the second stage, NIR image is introduced for NIR-assisted image denoising.
• Figure 5: The architecture of the proposed Frequency Dynamic Selection Mechanism (FDSM) and Frequency Exhaustive Fusion Mechanism (FEFM) for thorough frequency exploitation.