Hybrid Fusion: One-Minute Efficient Training for Zero-Shot Cross-Domain Image Fusion

TL;DR

A novel hybrid framework that achieves SOTA-comparable performance in about one minute on a RTX 4090 or two minutes on a consumer laptop GPU from scratch without any external model and demonstrates powerful zero-shot generalization across diverse tasks, from infrared-visible to medical imaging.

Abstract

Image fusion seeks to integrate complementary information from multiple sources into a single, superior image. While traditional methods are fast, they lack adaptability and performance. Conversely, deep learning approaches achieve state-of-the-art (SOTA) results but suffer from critical inefficiencies: their reliance on slow, resource-intensive, patch-based training introduces a significant gap with full-resolution inference. We propose a novel hybrid framework that resolves this trade-off. Our method utilizes a learnable U-Net to generate a dynamic guidance map that directs a classic, fixed Laplacian pyramid fusion kernel. This decoupling of policy learning from pixel synthesis enables remarkably efficient full-resolution training, eliminating the train-inference gap. Consequently, our model achieves SOTA-comparable performance in about one minute on a RTX 4090 or two minutes on a consumer laptop GPU from scratch without any external model and demonstrates powerful zero-shot generalization across diverse tasks, from infrared-visible to medical imaging. By design, the fused output is linearly constructed solely from source information, ensuring high faithfulness for critical applications. The codes are available at https://github.com/Zirconium233/HybridFusion

Figures (6)

• Figure 1: A conceptual comparison of image fusion paradigms.
• Figure 2: The architecture of our hybrid fusion model. A learnable U-Net (top) takes the concatenated visible luminance (Y) and infrared (Ir) channels to produce a guidance weight map. This map then directs a fixed, non-learnable Laplacian Pyramid fusion kernel (bottom) that operates on the multi-scale decompositions of the source images. The original chrominance (CbCr) is preserved and reapplied to the fused luminance, ensuring color faithfulness in the final output.
• Figure 3: Qualitative comparison on the MSRS dataset. Our method effectively highlights the pedestrian from the infrared image while preserving the textural details from the visible image. The learned weight map validates this interpretable behavior. For more comparisons, please refer to supplementary materials.
• Figure 4: Qualitative zero-shot comparison on medical tasks. Our MSRS-trained model produces faithful fusion results. Competitor methods like DTPF can exhibit textural and color artifacts when generalizing, while our in-domain trained models (e.g., Trained on SPECT) also show strong cross-domain transferability (e.g., to PET).
• Figure 5: Hyperparameter grid search for the unsupervised loss function, showing a broad region of optimal performance.