FusionRegister: Every Infrared and Visible Image Fusion Deserves Registration

TL;DR

A general cross-modality registration method guided by visual priors is proposed for infrared and visible image fusion task, termed FusionRegister, which achieves robustness by learning cross-modality misregistration representations rather than forcing alignment of all differences, ensuring stable outputs even under challenging input conditions.

Abstract

Spatial registration across different visual modalities is a critical but formidable step in multi-modality image fusion for real-world perception. Although several methods are proposed to address this issue, the existing registration-based fusion methods typically require extensive pre-registration operations, limiting their efficiency. To overcome these limitations, a general cross-modality registration method guided by visual priors is proposed for infrared and visible image fusion task, termed FusionRegister. Firstly, FusionRegister achieves robustness by learning cross-modality misregistration representations rather than forcing alignment of all differences, ensuring stable outputs even under challenging input conditions. Moreover, FusionRegister demonstrates strong generality by operating directly on fused results, where misregistration is explicitly represented and effectively handled, enabling seamless integration with diverse fusion methods while preserving their intrinsic properties. In addition, its efficiency is further enhanced by serving the backbone fusion method as a natural visual prior provider, which guides the registration process to focus only on mismatch regions, thereby avoiding redundant operations. Extensive experiments on three datasets demonstrate that FusionRegister not only inherits the fusion quality of state-of-the-art methods, but also delivers superior detail alignment and robustness, making it highly suitable for infrared and visible image fusion method. The code will be available at https://github.com/bociic/FusionRegister.

Figures (11)

• Figure 1: Existing registration-based fusion methods face three key limitations: (I) Depend on artificial deformations; (II) Cannot interact with fusion methods; (III) Require extensive pre-operations and global registration. We propose a visual prior-based post-registration method that leverages fusion results to preserve fusion quality while significantly enhancing structural accuracy, achieving efficiency and robustness.
• Figure 2: The fusion results show that even with spatial deformation in the infrared image, misregistration appears only in modality-shared regions, where structural similarity with the fine registered result is significantly reduced.
• Figure 3: The proposed FusionRegister consists of two core components: post-registration of misregistration regions via bi-directional warping, and a modality retainment block (MRB). Note that ML is misregistration location operation and e-w denotes element-wise.
• Figure 4: Detailed architecture of the Modality Retainment Block (MRB).
• Figure 5: Detailed architecture of the Correlation Layer.