Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SimpleFusion: A Simple Fusion Framework for Infrared and Visible Images

Ming Chen, Yuxuan Cheng, Xinwei He, Xinyue Wang, Yan Aze, Jinhai Xiang

TL;DR

This paper presents SimpleFusion, a simple yet effective framework for visible and infrared image fusion that follows the decompose-and-fusion paradigm, where the visible and the infrared images are decomposed into reflectance and illumination components via Retinex theory and followed by the fusion of these corresponding elements.

Abstract

Integrating visible and infrared images into one high-quality image, also known as visible and infrared image fusion, is a challenging yet critical task for many downstream vision tasks. Most existing works utilize pretrained deep neural networks or design sophisticated frameworks with strong priors for this task, which may be unsuitable or lack flexibility. This paper presents SimpleFusion, a simple yet effective framework for visible and infrared image fusion. Our framework follows the decompose-and-fusion paradigm, where the visible and the infrared images are decomposed into reflectance and illumination components via Retinex theory and followed by the fusion of these corresponding elements. The whole framework is designed with two plain convolutional neural networks without downsampling, which can perform image decomposition and fusion efficiently. Moreover, we introduce decomposition loss and a detail-to-semantic loss to preserve the complementary information between the two modalities for fusion. We conduct extensive experiments on the challenging benchmarks, verifying the superiority of our method over previous state-of-the-arts. Code is available at \href{https://github.com/hxwxss/SimpleFusion-A-Simple-Fusion-Framework-for-Infrared-and-Visible-Images}{https://github.com/hxwxss/SimpleFusion-A-Simple-Fusion-Framework-for-Infrared-and-Visible-Images}

SimpleFusion: A Simple Fusion Framework for Infrared and Visible Images

TL;DR

This paper presents SimpleFusion, a simple yet effective framework for visible and infrared image fusion that follows the decompose-and-fusion paradigm, where the visible and the infrared images are decomposed into reflectance and illumination components via Retinex theory and followed by the fusion of these corresponding elements.

Abstract

Integrating visible and infrared images into one high-quality image, also known as visible and infrared image fusion, is a challenging yet critical task for many downstream vision tasks. Most existing works utilize pretrained deep neural networks or design sophisticated frameworks with strong priors for this task, which may be unsuitable or lack flexibility. This paper presents SimpleFusion, a simple yet effective framework for visible and infrared image fusion. Our framework follows the decompose-and-fusion paradigm, where the visible and the infrared images are decomposed into reflectance and illumination components via Retinex theory and followed by the fusion of these corresponding elements. The whole framework is designed with two plain convolutional neural networks without downsampling, which can perform image decomposition and fusion efficiently. Moreover, we introduce decomposition loss and a detail-to-semantic loss to preserve the complementary information between the two modalities for fusion. We conduct extensive experiments on the challenging benchmarks, verifying the superiority of our method over previous state-of-the-arts. Code is available at \href{https://github.com/hxwxss/SimpleFusion-A-Simple-Fusion-Framework-for-Infrared-and-Visible-Images}{https://github.com/hxwxss/SimpleFusion-A-Simple-Fusion-Framework-for-Infrared-and-Visible-Images}
Paper Structure (15 sections, 14 equations, 2 figures, 3 tables)

This paper contains 15 sections, 14 equations, 2 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Method
  4. Problem Formulation and Challenges
  5. SimpleFusion
  6. Decomposition Loss
  7. Final Decomposition loss.
  8. Detail-to-Semantic Loss
  9. Overall Loss Function
  10. Experiments
  11. Experimental Setups
  12. Comparisons with State-of-the-arts
  13. Ablation study
  14. Conclusion
  15. Acknowledgement

Figures (2)

  • Figure 1: The framework of SimpleFusion. It takes infrared and visible images as input, which will be fed into a projection layer to remove unwanted features that are not considered in Retinex theory. For visible image, we decompose it with into illumination and reflectance components, while for infrared image we simply extract corresponding components for fusion, with plain CNNs. The final high-quality image is derived directly by composing the components via Retinex theory.
  • Figure 2: The typical fusion results on TNO ("man" image).