Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

IAIFNet: An Illumination-Aware Infrared and Visible Image Fusion Network

Qiao Yang, Yu Zhang, Zijing Zhao, Jian Zhang, Shunli Zhang

TL;DR

The paper tackles infrared-visible image fusion in low-light conditions by introducing IAIFNet, an illumination-aware framework that first estimates incident illumination maps and then fuses illumination-enhanced IR and visible images. The fusion network employs two novel components, Adaptive Differential Fusion Module (ADFM) and Salient Target Aware Module (STAM), to balance brightness and emphasize salient targets, operating in RGB/YCbCr space for color consistency. A Retinex-inspired illumination model and a composite loss (structure, intensity, gradient) guide the fusion toward high structural fidelity and reduced noise, achieving state-of-the-art results on LLVIP, with faster inference and smaller model size. The approach also improves downstream object detection performance, demonstrating practical impact for night-vision and surveillance tasks.

Abstract

Infrared and visible image fusion (IVIF) is used to generate fusion images with comprehensive features of both images, which is beneficial for downstream vision tasks. However, current methods rarely consider the illumination condition in low-light environments, and the targets in the fused images are often not prominent. To address the above issues, we propose an Illumination-Aware Infrared and Visible Image Fusion Network, named as IAIFNet. In our framework, an illumination enhancement network first estimates the incident illumination maps of input images. Afterwards, with the help of proposed adaptive differential fusion module (ADFM) and salient target aware module (STAM), an image fusion network effectively integrates the salient features of the illumination-enhanced infrared and visible images into a fusion image of high visual quality. Extensive experimental results verify that our method outperforms five state-of-the-art methods of fusing infrared and visible images.

IAIFNet: An Illumination-Aware Infrared and Visible Image Fusion Network

TL;DR

The paper tackles infrared-visible image fusion in low-light conditions by introducing IAIFNet, an illumination-aware framework that first estimates incident illumination maps and then fuses illumination-enhanced IR and visible images. The fusion network employs two novel components, Adaptive Differential Fusion Module (ADFM) and Salient Target Aware Module (STAM), to balance brightness and emphasize salient targets, operating in RGB/YCbCr space for color consistency. A Retinex-inspired illumination model and a composite loss (structure, intensity, gradient) guide the fusion toward high structural fidelity and reduced noise, achieving state-of-the-art results on LLVIP, with faster inference and smaller model size. The approach also improves downstream object detection performance, demonstrating practical impact for night-vision and surveillance tasks.

Abstract

Infrared and visible image fusion (IVIF) is used to generate fusion images with comprehensive features of both images, which is beneficial for downstream vision tasks. However, current methods rarely consider the illumination condition in low-light environments, and the targets in the fused images are often not prominent. To address the above issues, we propose an Illumination-Aware Infrared and Visible Image Fusion Network, named as IAIFNet. In our framework, an illumination enhancement network first estimates the incident illumination maps of input images. Afterwards, with the help of proposed adaptive differential fusion module (ADFM) and salient target aware module (STAM), an image fusion network effectively integrates the salient features of the illumination-enhanced infrared and visible images into a fusion image of high visual quality. Extensive experimental results verify that our method outperforms five state-of-the-art methods of fusing infrared and visible images.
Paper Structure (13 sections, 13 equations, 4 figures, 3 tables)

This paper contains 13 sections, 13 equations, 4 figures, 3 tables.

Table of Contents

  1. Introduction
  2. The Proposed Method
  3. Problem Formulation
  4. Overall Architecture
  5. Network Details
  6. Network Architecture
  7. Loss Function
  8. Experimental Results and Discussions
  9. Dataset and Implementation Details
  10. Performance Analysis
  11. Performance on Object Detection Task
  12. Ablation Analysis
  13. Conclusion

Figures (4)

  • Figure 1: Flowchart of our method. The left block is the overall framework of our IAIFNet, which mainly consists of two modules, i.e., an illumination enhancement network and an image fusion network. The right block illustrates the structure of ADFM, and the bottom shows the symbols' meanings.
  • Figure 2: Qualitative comparison of our IAIFNet and seven state-of-the-art methods on fusing three pairs of infrared and visible images (i.e., #060193, #120089 and #190001)). In each image, one red region is annotated for clear comparison.
  • Figure 3: Quantitative comparisons of different image fusion methods. A single point ($x,y$) on the curve denotes that there are ($100\times x$) % percent of image pairs that have metric values no more than y.
  • Figure 4: Qualitative comparisons of different strategies on #190271 and #010045, respectively.