FusionINN: Decomposable Image Fusion for Brain Tumor Monitoring

TL;DR

FusionINN tackles interpretability in multimodal brain MRI fusion by introducing an invertible normalizing-flow framework that jointly fuses two modalities into a fused image $y$ and a latent code $z$, while enabling exact decomposition back to the sources via $[\hat{x}_1,\hat{x}_2] = f^{-1}_{\theta}(y,z)$. The model is trained in a fully unsupervised manner using a fusion loss, a latent regularization term, and a decomposition loss, and is equipped with a CNN-based invertible coupling architecture and inverse down-/up-sampling to capture multi-scale features. On BraTS-2018 data, FusionINN achieves competitive or superior fusion metrics and demonstrates accurate decomposition, with demonstrated generalization to unseen clinical modalities such as DWI-ADC, highlighting potential clinical impact for interpretable, real-time brain tumor monitoring. The work opens avenues for clinically oriented fusion by enabling decomposition into source modalities, potentially aiding segmentation and decision-making, with code made available for reproducibility and further development.

Abstract

Image fusion typically employs non-invertible neural networks to merge multiple source images into a single fused image. However, for clinical experts, solely relying on fused images may be insufficient for making diagnostic decisions, as the fusion mechanism blends features from source images, thereby making it difficult to interpret the underlying tumor pathology. We introduce FusionINN, a novel decomposable image fusion framework, capable of efficiently generating fused images and also decomposing them back to the source images. FusionINN is designed to be bijective by including a latent image alongside the fused image, while ensuring minimal transfer of information from the source images to the latent representation. To the best of our knowledge, we are the first to investigate the decomposability of fused images, which is particularly crucial for life-sensitive applications such as medical image fusion compared to other tasks like multi-focus or multi-exposure image fusion. Our extensive experimentation validates FusionINN over existing discriminative and generative fusion methods, both subjectively and objectively. Moreover, compared to a recent denoising diffusion-based fusion model, our approach offers faster and qualitatively better fusion results.

Figures (6)

• Figure 1: An illustration of the task of image fusion and decomposition.
• Figure 2: An overview of the FusionINN framework.
• Figure 3: Loss curves for a FusionINN training instance with $k = 4$, $\lambda = 0.9$ and $\alpha = 0.5$.
• Figure 4: Fusion results obtained from the evaluated models on a sample validation image pair. The $Q_{SSIM}$ scores for individual modalities are shown in the fused images.
• Figure 5: FusionINN results at $\alpha = 0.5$, $z \sim \mathcal{N}(0, I)$ and $k$ as number of coupling blocks.