Multi-Modal Image Fusion via Intervention-Stable Feature Learning

Abstract

Multi-modal image fusion integrates complementary information from different modalities into a unified representation. Current methods predominantly optimize statistical correlations between modalities, often capturing dataset-induced spurious associations that degrade under distribution shifts. In this paper, we propose an intervention-based framework inspired by causal principles to identify robust cross-modal dependencies. Drawing insights from Pearl's causal hierarchy, we design three principled intervention strategies to probe different aspects of modal relationships: i) complementary masking with spatially disjoint perturbations tests whether modalities can genuinely compensate for each other's missing information, ii) random masking of identical regions identifies feature subsets that remain informative under partial observability, and iii) modality dropout evaluates the irreplaceable contribution of each modality. Based on these interventions, we introduce a Causal Feature Integrator (CFI) that learns to identify and prioritize intervention-stable features maintaining importance across different perturbation patterns through adaptive invariance gating, thereby capturing robust modal dependencies rather than spurious correlations. Extensive experiments demonstrate that our method achieves SOTA performance on both public benchmarks and downstream high-level vision tasks.

Figures (8)

• Figure 1: Comparison with SOTA method in training framework and performance. (a) General designs of existing methods, which often rely on empirical trial-and-error to fit all source features. (b) Our framework actively probes modal dependencies through structured interventions. (c) The superiority of our method is validated by its advantages in static metrics (CC, PSNR), efficiency analysis (FPS), semantic segmentation (mIoU), object detection (mAP), and qualitative comparisons.
• Figure 2: The framework of the proposed method. It employs a U-Net-like Siamese architecture that incorporates CFI within the decoder to enable robust multi-modal fusion. By leveraging three complementary intervention strategies, the model learns to identify intervention-stable features that represent genuine cross-modal complementarity rather than spurious statistical co-occurrences arising from dataset biases.
• Figure 3: An illustrative example of feature visualization. Through the learnable invariance gates, the model progressively focuses on intervention-stable features across successive iterations, prioritizing regions that maintain consistent importance under perturbations to ensure effective integration of complementary modalities.
• Figure 4: Qualitative comparison with SOTA methods on the IVIF benchmarks.
• Figure 5: Visualization comparison of ablation study