Spatial-frequency Dual-Domain Feature Fusion Network for Low-Light Remote Sensing Image Enhancement

TL;DR

This work tackles the challenge of enhancing low-light remote sensing images by decoupling degradation into amplitude illumination and phase refinement tasks. It introduces a Dual-Domain Feature Fusion Network (DFFN) that integrates spatial and Fourier-domain information via Dual-Domain Amplitude Block (DDAB), Dual-Domain Phase Block (DDPB), and an Information Fusion Affine Module (IFAM) to enable cross-stage, cross-scale feature interaction. The approach demonstrates strong performance against state-of-the-art methods on synthetic and real nocturnal RS datasets, achieving favorable accuracy with a mindful parameter budget. The work also provides two new datasets, iSAID-dark and darkrs, to support supervised training and robust testing in low-light remote sensing scenarios.

Abstract

Low-light remote sensing images generally feature high resolution and high spatial complexity, with continuously distributed surface features in space. This continuity in scenes leads to extensive long-range correlations in spatial domains within remote sensing images. Convolutional Neural Networks, which rely on local correlations for long-distance modeling, struggle to establish long-range correlations in such images. On the other hand, transformer-based methods that focus on global information face high computational complexities when processing high-resolution remote sensing images. From another perspective, Fourier transform can compute global information without introducing a large number of parameters, enabling the network to more efficiently capture the overall image structure and establish long-range correlations. Therefore, we propose a Dual-Domain Feature Fusion Network (DFFN) for low-light remote sensing image enhancement. Specifically, this challenging task of low-light enhancement is divided into two more manageable sub-tasks: the first phase learns amplitude information to restore image brightness, and the second phase learns phase information to refine details. To facilitate information exchange between the two phases, we designed an information fusion affine block that combines data from different phases and scales. Additionally, we have constructed two dark light remote sensing datasets to address the current lack of datasets in dark light remote sensing image enhancement. Extensive evaluations show that our method outperforms existing state-of-the-art methods. The code is available at https://github.com/iijjlk/DFFN.

Figures (18)

• Figure 1: Comparison between the latest state-of-the-art methods and our approach. As shown in the zoomed-in region, these methods exhibit color distortion and excessive noise, leading to reduced visual quality. In contrast, our method presents vibrant colors and sharp outlines. Transformer-based method SNRxu2022snr achieves similar visual effects to ours but with 40.1M parameters. Additionally, in (b), we compute the NIQEmittal2012making scores on a no-reference dataset and display them in inverse form. Furthermore, in (c), scores of SSIMwang2004image and PSNRhuynh2008scope metrics on a reference dataset are shown. It can be easily observed that our method significantly outperforms others.
• Figure 2: Our Motivation. As observed in the figure, when swapping the amplitude and phase of low-light images with those of normal images, the overall ambient light of the image containing degraded amplitude darkens while maintaining clear details. On the other hand, in the image containing phase degradation, there is no change in brightness, but the image details vanish. This indicates that Fourier transform can decouple degradation information.
• Figure 3: Overall network structure diagram. The specific structures of DDAB and DDPB are displayed below the diagram. The low-light image $I_{low}$ first undergoes an amplitude illumination stage to obtain a preliminary enhanced image $O_A$, followed by a phase refinement stage to obtain the final enhanced image $O_P$. In this process, the feature information from different stages is passed into IFAM for mapping to enhance the contextual representation capability of the model.
• Figure 4: The specific structure of the proposed IFAM consists of two key components: the Information Fusion Module and the Information Mapping Module. $U_{i}$ represents the feature obtained after the $i$-th upsampling in the Phase refinement stage.
• Figure 5: Samples from the proposed iSAID-dark(Up) and darkrs(Down) dataset.