Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Direction-aware multi-scale gradient loss for infrared and visible image fusion

Kaixuan Yang, Wei Xiang, Zhenshuai Chen, Tong Jin, Yunpeng Liu

TL;DR

This paper tackles the challenge of edge preservation in infrared–visible image fusion by identifying that traditional gradient losses discard directional information, leading to suboptimal fusion. It introduces a direction-aware, multi-scale gradient loss that axis‑wisely selects the dominant gradient per axis from the visible and infrared inputs and supervises the fused image's gradient vector across multiple scales. Across four public benchmarks (MSRS, FMB, M3FD, LLVIP) and the ReCoNet baseline, the proposed loss consistently improves information content, structural fidelity, and perceptual quality, while reducing gradient cancellation artifacts. The approach is plug‑and‑play, does not require architectural changes, and demonstrates robust generalization to unseen data, highlighting its practical value for improving IVIF systems in varied conditions.

Abstract

Infrared and visible image fusion aims to integrate complementary information from co-registered source images to produce a single, informative result. Most learning-based approaches train with a combination of structural similarity loss, intensity reconstruction loss, and a gradient-magnitude term. However, collapsing gradients to their magnitude removes directional information, yielding ambiguous supervision and suboptimal edge fidelity. We introduce a direction-aware, multi-scale gradient loss that supervises horizontal and vertical components separately and preserves their sign across scales. This axis-wise, sign-preserving objective provides clear directional guidance at both fine and coarse resolutions, promoting sharper, better-aligned edges and richer texture preservation without changing model architectures or training protocols. Experiments on open-source model and multiple public benchmarks demonstrate effectiveness of our approach.

Direction-aware multi-scale gradient loss for infrared and visible image fusion

TL;DR

This paper tackles the challenge of edge preservation in infrared–visible image fusion by identifying that traditional gradient losses discard directional information, leading to suboptimal fusion. It introduces a direction-aware, multi-scale gradient loss that axis‑wisely selects the dominant gradient per axis from the visible and infrared inputs and supervises the fused image's gradient vector across multiple scales. Across four public benchmarks (MSRS, FMB, M3FD, LLVIP) and the ReCoNet baseline, the proposed loss consistently improves information content, structural fidelity, and perceptual quality, while reducing gradient cancellation artifacts. The approach is plug‑and‑play, does not require architectural changes, and demonstrates robust generalization to unseen data, highlighting its practical value for improving IVIF systems in varied conditions.

Abstract

Infrared and visible image fusion aims to integrate complementary information from co-registered source images to produce a single, informative result. Most learning-based approaches train with a combination of structural similarity loss, intensity reconstruction loss, and a gradient-magnitude term. However, collapsing gradients to their magnitude removes directional information, yielding ambiguous supervision and suboptimal edge fidelity. We introduce a direction-aware, multi-scale gradient loss that supervises horizontal and vertical components separately and preserves their sign across scales. This axis-wise, sign-preserving objective provides clear directional guidance at both fine and coarse resolutions, promoting sharper, better-aligned edges and richer texture preservation without changing model architectures or training protocols. Experiments on open-source model and multiple public benchmarks demonstrate effectiveness of our approach.
Paper Structure (17 sections, 19 equations, 2 figures, 5 tables)

This paper contains 17 sections, 19 equations, 2 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Infrared and visible image fusion
  4. Loss for IVIF
  5. Method
  6. Notation
  7. Baseline gradient loss
  8. Signed gradient loss from TC-MoA
  9. Multi-scale gradient loss with direction alignment
  10. Experiment results
  11. Evaluation metrics
  12. MSRS results
  13. FMB results
  14. M3FD results
  15. LLVIP results
  16. ...and 2 more sections

Figures (2)

  • Figure 1: Comparison of gradient loss formulations. The proposed direction-aligned loss (c) performs independent supervision along horizontal and vertical directions, avoiding gradient cancellation and preserving sharper edge structures.
  • Figure 2: Qualitative comparison on the MSRS dataset. From left to right: (a) infrared image, (b) visible image, (c) ORI, (d) GRAD, (e) TCMoA, and (f) Ours.