AngularFuse: A Closer Look at Angle-based Perception for Spatial-Sensitive Multi-Modality Image Fusion

TL;DR

AngularFuse tackles unsupervised visible–infrared image fusion by addressing limitations of pixel-wise intensity losses and gradient-only losses through an angle-aware perception framework. It introduces three components: Complementary Mask Generation to enforce cross-modal complementarity, Fine-Grained Reference Image Synthesis to produce richer, balanced references for supervision, and an angle-aware gradient loss to enforce consistency in both gradient magnitude and direction. Across MSRS, RoadScene, and M3FD, AngularFuse achieves state-of-the-art fusion metrics and demonstrates strong generalization in OOD scenarios, while maintaining efficiency. This work advances practical VIF by integrating cross-modal completion, texture-focused reference supervision, and gradient-vector reasoning.

Abstract

Visible-infrared image fusion is crucial in key applications such as autonomous driving and nighttime surveillance. Its main goal is to integrate multimodal information to produce enhanced images that are better suited for downstream tasks. Although deep learning based fusion methods have made significant progress, mainstream unsupervised approaches still face serious challenges in practical applications. Existing methods mostly rely on manually designed loss functions to guide the fusion process. However, these loss functions have obvious limitations. On one hand, the reference images constructed by existing methods often lack details and have uneven brightness. On the other hand, the widely used gradient losses focus only on gradient magnitude. To address these challenges, this paper proposes an angle-based perception framework for spatial-sensitive image fusion (AngularFuse). At first, we design a cross-modal complementary mask module to force the network to learn complementary information between modalities. Then, a fine-grained reference image synthesis strategy is introduced. By combining Laplacian edge enhancement with adaptive histogram equalization, reference images with richer details and more balanced brightness are generated. Last but not least, we introduce an angle-aware loss, which for the first time constrains both gradient magnitude and direction simultaneously in the gradient domain. AngularFuse ensures that the fused images preserve both texture intensity and correct edge orientation. Comprehensive experiments on the MSRS, RoadScene, and M3FD public datasets show that AngularFuse outperforms existing mainstream methods with clear margin. Visual comparisons further confirm that our method produces sharper and more detailed results in challenging scenes, demonstrating superior fusion capability.

Figures (8)

• Figure 2: The proposed Complementary Mask Generation (ComMask) module. First, a patch is randomly selected from the input infrared image $I_{ir}$ and the visible image $I_{vi}$. Then a complementary mask is applied to perform complementary occlusion, producing $I_{ir}^{cm}$ and $I_{vi}^{cm}$ with complementary spatial structures, for a cross-modality self-supervised learning.
• Figure 3: The fine-grained reference image synthesis process. First, Laplacian convolution is applied to the infrared and visible images to extract the edge map $I_{edge}$. Then the high-intensity features $I_{max}$ are combined with $I_{edge}$ to obtain the enhanced image $I_{en}$ with rich textures. Finally, an equalization operation is applied to generate $I_{eq}$ with uniform brightness.
• Figure 4: The computation flow of the angle-aware loss. First, the Sobel operator is applied to obtain the gradient magnitude and direction of both the fused image and the reference image. Then the maximum-magnitude preservation branch generates the reference gradient $Grad_{ref}$. Finally, a loss that constrains both magnitude and direction is defined to ensure that the fused result maintains better consistency with the source images in texture details.
• Figure 5: Fusion visualization results on the MSRS test set
• Figure 6: Fusion visualization results on the M3FD test set. In complex urban scenes with strong light sources, our method highlights thermal targets while preserving texture details.