Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DAF-Net: A Dual-Branch Feature Decomposition Fusion Network with Domain Adaptive for Infrared and Visible Image Fusion

Jian Xu, Xin He

TL;DR

A dual-branch feature decomposition fusion network (DAF-Net) with domain adaptive is proposed, which introduces Multi-Kernel Maximum Mean Discrepancy (MK-MMD) into the base encoder and designs a hybrid kernel function suitable for infrared and visible image fusion.

Abstract

Infrared and visible image fusion aims to combine complementary information from both modalities to provide a more comprehensive scene understanding. However, due to the significant differences between the two modalities, preserving key features during the fusion process remains a challenge. To address this issue, we propose a dual-branch feature decomposition fusion network (DAF-Net) with domain adaptive, which introduces Multi-Kernel Maximum Mean Discrepancy (MK-MMD) into the base encoder and designs a hybrid kernel function suitable for infrared and visible image fusion. The base encoder built on the Restormer network captures global structural information while the detail encoder based on Invertible Neural Networks (INN) focuses on extracting detail texture information. By incorporating MK-MMD, the DAF-Net effectively aligns the latent feature spaces of visible and infrared images, thereby improving the quality of the fused images. Experimental results demonstrate that the proposed method outperforms existing techniques across multiple datasets, significantly enhancing both visual quality and fusion performance. The related Python code is available at https://github.com/xujian000/DAF-Net.

DAF-Net: A Dual-Branch Feature Decomposition Fusion Network with Domain Adaptive for Infrared and Visible Image Fusion

TL;DR

A dual-branch feature decomposition fusion network (DAF-Net) with domain adaptive is proposed, which introduces Multi-Kernel Maximum Mean Discrepancy (MK-MMD) into the base encoder and designs a hybrid kernel function suitable for infrared and visible image fusion.

Abstract

Infrared and visible image fusion aims to combine complementary information from both modalities to provide a more comprehensive scene understanding. However, due to the significant differences between the two modalities, preserving key features during the fusion process remains a challenge. To address this issue, we propose a dual-branch feature decomposition fusion network (DAF-Net) with domain adaptive, which introduces Multi-Kernel Maximum Mean Discrepancy (MK-MMD) into the base encoder and designs a hybrid kernel function suitable for infrared and visible image fusion. The base encoder built on the Restormer network captures global structural information while the detail encoder based on Invertible Neural Networks (INN) focuses on extracting detail texture information. By incorporating MK-MMD, the DAF-Net effectively aligns the latent feature spaces of visible and infrared images, thereby improving the quality of the fused images. Experimental results demonstrate that the proposed method outperforms existing techniques across multiple datasets, significantly enhancing both visual quality and fusion performance. The related Python code is available at https://github.com/xujian000/DAF-Net.
Paper Structure (17 sections, 17 equations, 4 figures, 2 tables)

This paper contains 17 sections, 17 equations, 4 figures, 2 tables.

Table of Contents

  1. Introduction
  2. PROPOSED METHOD
  3. Network Architecture
  4. The encoder-decoder branches
  5. The domain adaptive layer
  6. Two-stage training
  7. Stage I (Encoder-decoder branches training)
  8. Stage II (Fusing layer training)
  9. Loss Function
  10. Stage I (Encoder-decoder branches training)
  11. Stage II (Fusing layer training)
  12. EXPERIMENTS AND RESULTS
  13. Experimental setup
  14. Implement details
  15. Qualitative Results
  16. ...and 2 more sections

Figures (4)

  • Figure 1: The framwork of the proposed DAF-Net.
  • Figure 2: Codec branch training process, which there is a domain adaptive layer between the basic encoders of infrared images and visible images.
  • Figure 3: The process of fusing layer training.
  • Figure 4: Comparison of results for Infrared-Visible Image Fusion task.