GTFMN: Guided Texture and Feature Modulation Network for Low-Light Image Enhancement and Super-Resolution

TL;DR

This paper tackles the problem of simultaneous low-light enhancement and image super-resolution, where illumination degradation and downsampling co-occur. It introduces GTFMN, a dual-stream network that decouples illumination estimation from texture restoration; the Illumination Stream predicts a spatially varying map M and the Texture Stream uses Illumination Guided Modulation Blocks to adaptively modulate features based on M. Key contributions include the decoupled illumination estimation, the novel IGM Block that blends self-attention with illumination-guided attention, and a lightweight architecture that achieves state-of-the-art results on OmniNormal5/15 with about 8.78 million parameters. Results show improved quantitative metrics and visual quality, demonstrating that explicit illumination guidance improves region-wise restoration in challenging low-light scenes.

Abstract

Low-light image super-resolution (LLSR) is a challenging task due to the coupled degradation of low resolution and poor illumination. To address this, we propose the Guided Texture and Feature Modulation Network (GTFMN), a novel framework that decouples the LLSR task into two sub-problems: illumination estimation and texture restoration. First, our network employs a dedicated Illumination Stream whose purpose is to predict a spatially varying illumination map that accurately captures lighting distribution. Further, this map is utilized as an explicit guide within our novel Illumination Guided Modulation Block (IGM Block) to dynamically modulate features in the Texture Stream. This mechanism achieves spatially adaptive restoration, enabling the network to intensify enhancement in poorly lit regions while preserving details in well-exposed areas. Extensive experiments demonstrate that GTFMN achieves the best performance among competing methods on the OmniNormal5 and OmniNormal15 datasets, outperforming them in both quantitative metrics and visual quality.

Figures (4)

• Figure 1: The overall framework and core concept of GTFMN.
• Figure 2: The detailed architecture of our GTFMN. The network consists of two main branches. The Illumination Stream (left) takes the low-resolution input ($I_{LR}$) and employs a decoupled design with a Structure Decoder and a Global Predictor to estimate a stable illumination map ($\mathbf{M}$). The Texture Stream (right) processes the image features through a series of Illumination-Guided Feature Modulation Blocks (IGM Blocks). Within each IGM Block, the $\mathbf{M}$ generates a guided attention map ($\mathbf{A}_{\text{guide }}$), which is fused with the feature-derived self-attention map ($\mathbf{A}_{\text{self}}$) to dynamically modulate the texture features. Finally, a reconstruction module produces the super-resolution image ($I_{SR}$).
• Figure 3: Qualitative comparison with state-of-the-art methods on the OmniNormal15 dataset for $\times2$ and $\times4$ SR.
• Figure 4: Ablation study on the effectiveness of the Illumination Stream.