Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Dynamic Brightness Adaptation for Robust Multi-modal Image Fusion

Yiming Sun, Bing Cao, Pengfei Zhu, Qinghua Hu

TL;DR

The Brightness Adaptive multimodal dynamic fusion framework (BA-Fusion) is proposed, which achieves robust image fusion despite dynamic brightness fluctuations, and surpasses state-of-the-art methods in preserving multi-modal image information and visual fidelity, while exhibiting remarkable robustness across varying brightness levels.

Abstract

Infrared and visible image fusion aim to integrate modality strengths for visually enhanced, informative images. Visible imaging in real-world scenarios is susceptible to dynamic environmental brightness fluctuations, leading to texture degradation. Existing fusion methods lack robustness against such brightness perturbations, significantly compromising the visual fidelity of the fused imagery. To address this challenge, we propose the Brightness Adaptive multimodal dynamic fusion framework (BA-Fusion), which achieves robust image fusion despite dynamic brightness fluctuations. Specifically, we introduce a Brightness Adaptive Gate (BAG) module, which is designed to dynamically select features from brightness-related channels for normalization, while preserving brightness-independent structural information within the source images. Furthermore, we propose a brightness consistency loss function to optimize the BAG module. The entire framework is tuned via alternating training strategies. Extensive experiments validate that our method surpasses state-of-the-art methods in preserving multi-modal image information and visual fidelity, while exhibiting remarkable robustness across varying brightness levels. Our code is available: https://github.com/SunYM2020/BA-Fusion.

Dynamic Brightness Adaptation for Robust Multi-modal Image Fusion

TL;DR

The Brightness Adaptive multimodal dynamic fusion framework (BA-Fusion) is proposed, which achieves robust image fusion despite dynamic brightness fluctuations, and surpasses state-of-the-art methods in preserving multi-modal image information and visual fidelity, while exhibiting remarkable robustness across varying brightness levels.

Abstract

Infrared and visible image fusion aim to integrate modality strengths for visually enhanced, informative images. Visible imaging in real-world scenarios is susceptible to dynamic environmental brightness fluctuations, leading to texture degradation. Existing fusion methods lack robustness against such brightness perturbations, significantly compromising the visual fidelity of the fused imagery. To address this challenge, we propose the Brightness Adaptive multimodal dynamic fusion framework (BA-Fusion), which achieves robust image fusion despite dynamic brightness fluctuations. Specifically, we introduce a Brightness Adaptive Gate (BAG) module, which is designed to dynamically select features from brightness-related channels for normalization, while preserving brightness-independent structural information within the source images. Furthermore, we propose a brightness consistency loss function to optimize the BAG module. The entire framework is tuned via alternating training strategies. Extensive experiments validate that our method surpasses state-of-the-art methods in preserving multi-modal image information and visual fidelity, while exhibiting remarkable robustness across varying brightness levels. Our code is available: https://github.com/SunYM2020/BA-Fusion.

Paper Structure

This paper contains 31 sections, 8 equations, 6 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Infrared and Visible Image Fusion
  4. Brightness Correction
  5. Method
  6. Overall Architecture
  7. Brightness Adaptive Gate
  8. Brightness Normalization
  9. Dynamic Gating Module
  10. Alternating Training Strategy
  11. Fusion Loss
  12. Brightness Consistency Loss
  13. Experiments
  14. Experimental Setting
  15. Datasets and Partition Protocol.
  16. ...and 16 more sections

Figures (6)

  • Figure 1: Visual comparisons on fused results and pixel histograms under dynamic brightness conditions. Our method keeps robust performance under varying levels of brightness.
  • Figure 2: The architecture of BA-Fusion. BA-Fusion consists of a Brightness Adaptive Gate (BAG), and the multimodal fusion backbone network.
  • Figure 3: Qualitative comparisons of various methods on representative images selected from the LLVIP dataset.
  • Figure 4: Qualitative comparisons of various methods on representative images selected from the M$^{3}$FD dataset.
  • Figure 5: Channel visualization on dynamic brightness conditions.
  • ...and 1 more figures