W-DUALMINE: Reliability-Weighted Dual-Expert Fusion With Residual Correlation Preservation for Medical Image Fusion

TL;DR

Medical image fusion aims to combine complementary information from multiple modalities, but existing methods often trade off global statistical similarity against local detail. W-DUALMINE introduces reliability-weighted dual-expert fusion with a residual-to-average fusion paradigm, using dense reliability maps, a global-context spatial expert, a wavelet frequency expert, and a soft gradient mixer to preserve both CC and MI. The approach is reinforced by a composite loss including Average Content, Gradient Max, Explicit Correlation, InfoNCE MI proxy, and Reconstruction terms, jointly optimizing global consistency and local detail. Experiments on CT-MRI, PET-MRI, and SPECT-MRI demonstrate consistent improvements over AdaFuse and ASFE-Fusion in MI, CC, and PSNR/FMI, indicating robust performance across modality pairs and noise conditions.

Abstract

Medical image fusion integrates complementary information from multiple imaging modalities to improve clinical interpretation. However, existing deep learningbased methods, including recent spatial-frequency frameworks such as AdaFuse and ASFE-Fusion, often suffer from a fundamental trade-off between global statistical similaritymeasured by correlation coefficient (CC) and mutual information (MI)and local structural fidelity. This paper proposes W-DUALMINE, a reliability-weighted dual-expert fusion framework designed to explicitly resolve this trade-off through architectural constraints and a theoretically grounded loss design. The proposed method introduces dense reliability maps for adaptive modality weighting, a dual-expert fusion strategy combining a global-context spatial expert and a wavelet-domain frequency expert, and a soft gradient-based arbitration mechanism. Furthermore, we employ a residual-to-average fusion paradigm that guarantees the preservation of global correlation while enhancing local details. Extensive experiments on CT-MRI, PET-MRI, and SPECT-MRI datasets demonstrate that W-DUALMINE consistently outperforms AdaFuse and ASFE-Fusion in CC and MI metrics while

Figures (6)

• Figure 1: Architecture of the Reliability-Weighted Dual-Expert Fusion Network. The framework processes multi-modal inputs (e.g., CT/PET and MRI) through a Siamese encoder composed of ResBlocks. Feature maps are weighted by a Reliability Estimation module before entering two parallel expert branches: (1) A Global Context Expert that utilizes dilated convolutions to capture multi-scale spatial details, and (2) A Wavelet Expert that performs frequency-domain fusion using Discrete Wavelet Transform (DWT). The outputs of these experts are fused by a Soft Gradient Mixer, which employs an edge-aware attention mechanism based on gradient magnitudes ($|\nabla S|, |\nabla W|$). Finally, the fused features are decoded into a residual map and added to the averaged input base to reconstruct the final fused image.
• Figure 2: The overall framework of W-DUALMINE. The architecture consists of Siamese encoders extracting multi-scale features, which are fused and projected for contrastive learning. The network is optimized via a composite loss function comprising: (1) Average Content Loss ($\mathcal{L}_{avg}$) for low-frequency consistency, (2) Gradient-Max Loss ($\mathcal{L}_{grad}$) for high-frequency edge preservation, (3) Correlation Loss ($\mathcal{L}_{cc}$) to enforce linearity, (4) Mutual Information Loss ($\mathcal{L}_{mi}$) in the feature space, and (5) Reconstruction Loss ($\mathcal{L}_{rec}$) for domain fidelity.
• Figure 3: Qualitative comparison on the CT--MRI fusion task. From left to right: CT image, MRI image, ASFE-Fusion, AdaFuse, and W-DUALMINE. Red boxes highlight regions of interest for detailed comparison. The proposed method preserves sharp CT structures while maintaining MRI soft-tissue contrast.
• Figure 4: Qualitative comparison on the PET--MRI fusion task. From left to right: PET image, MRI image, ASFE-Fusion, AdaFuse, and W-DUALMINE. Highlighted regions demonstrate superior preservation of functional activity without color diffusion.
• Figure 5: Qualitative comparison on the SPECT--MRI fusion task. From left to right: SPECT image, MRI image, ASFE-Fusion, AdaFuse, and W-DUALMINE. The proposed method maintains low-contrast functional regions while preserving anatomical structure.