Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Quaternion Infrared Visible Image Fusion

Weihua Yang, Yicong Zhou

TL;DR

A quaternion low-visibility feature learning model to adaptively extract salient thermal targets and fine-grained texture details from input infrared and visible images respectively under diverse degraded conditions and a quaternion adaptive unsharp masking method to adaptively improve high-frequency feature enhancement with balanced illumination.

Abstract

Visible images provide rich details and color information only under well-lighted conditions while infrared images effectively highlight thermal targets under challenging conditions such as low visibility and adverse weather. Infrared-visible image fusion aims to integrate complementary information from infrared and visible images to generate a high-quality fused image. Existing methods exhibit critical limitations such as neglecting color structure information in visible images and performance degradation when processing low-quality color-visible inputs. To address these issues, we propose a quaternion infrared-visible image fusion (QIVIF) framework to generate high-quality fused images completely in the quaternion domain. QIVIF proposes a quaternion low-visibility feature learning model to adaptively extract salient thermal targets and fine-grained texture details from input infrared and visible images respectively under diverse degraded conditions. QIVIF then develops a quaternion adaptive unsharp masking method to adaptively improve high-frequency feature enhancement with balanced illumination. QIVIF further proposes a quaternion hierarchical Bayesian fusion model to integrate infrared saliency and enhanced visible details to obtain high-quality fused images. Extensive experiments across diverse datasets demonstrate that our QIVIF surpasses state-of-the-art methods under challenging low-visibility conditions.

Quaternion Infrared Visible Image Fusion

TL;DR

A quaternion low-visibility feature learning model to adaptively extract salient thermal targets and fine-grained texture details from input infrared and visible images respectively under diverse degraded conditions and a quaternion adaptive unsharp masking method to adaptively improve high-frequency feature enhancement with balanced illumination.

Abstract

Visible images provide rich details and color information only under well-lighted conditions while infrared images effectively highlight thermal targets under challenging conditions such as low visibility and adverse weather. Infrared-visible image fusion aims to integrate complementary information from infrared and visible images to generate a high-quality fused image. Existing methods exhibit critical limitations such as neglecting color structure information in visible images and performance degradation when processing low-quality color-visible inputs. To address these issues, we propose a quaternion infrared-visible image fusion (QIVIF) framework to generate high-quality fused images completely in the quaternion domain. QIVIF proposes a quaternion low-visibility feature learning model to adaptively extract salient thermal targets and fine-grained texture details from input infrared and visible images respectively under diverse degraded conditions. QIVIF then develops a quaternion adaptive unsharp masking method to adaptively improve high-frequency feature enhancement with balanced illumination. QIVIF further proposes a quaternion hierarchical Bayesian fusion model to integrate infrared saliency and enhanced visible details to obtain high-quality fused images. Extensive experiments across diverse datasets demonstrate that our QIVIF surpasses state-of-the-art methods under challenging low-visibility conditions.
Paper Structure (20 sections, 5 theorems, 33 equations, 10 figures, 2 tables, 1 algorithm)

This paper contains 20 sections, 5 theorems, 33 equations, 10 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Quaternion algebra and representation
  4. Quaternion representation
  5. Quaternion low rank regularization
  6. Proposed Framework
  7. Overview
  8. Quaternion low-visibility feature learning
  9. Quaternion lighting suppression
  10. Quaternion low-rank decomposition
  11. Overall model formulation
  12. Optimization
  13. Quaternion adaptive unsharp masking
  14. Quaternion Hierarchical Bayesian fusion
  15. Experiments
  16. ...and 5 more sections

Key Result

Lemma 1

(Quaternion weighted Schatten-$p$ norm miao2020quaternion) For any $\lambda \geq 0$, quaternion matrix $\dot{\boldsymbol{\mathrm{Y}}}$ and $\dot{\boldsymbol{\mathrm{X}}}$$\in \mathbbm{H}^{H\times W}$ with the rank of $r$, then the quaternion weighted Schatten p-norm problem ($0<p<1$) can be defined with the given $p$ and $w$, there exists a specific threshold ($p<1$) ${\tau}_p^{GST}(\lambda w)={(

Figures (10)

  • Figure 1: Samples pairs of the infrared and the visible from the challenging low-visibility scenarios including glow effects, color shifts, haze, low light and over-exposure conditions.
  • Figure 2: Visual Comparison of various IVIF algorithms in normal and low visibility conditions. VIS denotes the visible images. LatLRR wang2020latent and AVSHB fu2022adaptive correspond to traditional model-based IVIF methods. SHIP zheng2024probing and DAFusion wang2025degradation are deep-learning-based methods. Compared to deep-learning-based methods, LatLRR and AVSHB inject more grayscale pixels of the infrared and exhibit a low-contrast background particular in the low-visibility condition.
  • Figure 3: Quaternion representation of a color image. We encode the three channels of the color image into a pure quaternion matrix.
  • Figure 4: Flowchart of our quaternion infrared visible image fusion.
  • Figure 5: Decomposition results of the infrared and the visible using QLRD model. The QLRD model decomposes the given input quaternion representation into a smooth structure feature layer and a salient target feature layer with rich details.
  • ...and 5 more figures

Theorems & Definitions (5)

  • Lemma 1
  • Lemma 2
  • Lemma 3
  • Theorem 1
  • Lemma 4