Med-K2N: Flexible K-to-N Modality Translation for Medical Image Synthesis

TL;DR

Med-K2N tackles flexible cross-modal medical image synthesis by enabling $K→N$ mappings through a quality-aware, progressive fusion framework. It couples a MultiScaleNet-based feature extractor with three collaborative fusion modules (PreWeightNet, ThresholdNet, EffiWeightNet) and a TaskHeadNet, plus the CMIM to enforce modality identity using vision-language cues. Across BraTS-derived and ISLES 2022 datasets, it achieves significant PSNR/SSIM gains over state-of-the-art methods and demonstrates robust improvements across varying numbers of input modalities. The approach enables robust, controllable generation across modalities while mitigating information dilution and modality-confusion, with potential to streamline clinical workflows if validated clinically.

Abstract

Cross-modal medical image synthesis research focuses on reconstructing missing imaging modalities from available ones to support clinical diagnosis. Driven by clinical necessities for flexible modality reconstruction, we explore K to N medical generation, where three critical challenges emerge: How can we model the heterogeneous contributions of different modalities to various target tasks? How can we ensure fusion quality control to prevent degradation from noisy information? How can we maintain modality identity consistency in multi-output generation? Driven by these clinical necessities, and drawing inspiration from SAM2's sequential frame paradigm and clinicians' progressive workflow of incrementally adding and selectively integrating multi-modal information, we treat multi-modal medical data as sequential frames with quality-driven selection mechanisms. Our key idea is to "learn" adaptive weights for each modality-task pair and "memorize" beneficial fusion patterns through progressive enhancement. To achieve this, we design three collaborative modules: PreWeightNet for global contribution assessment, ThresholdNet for adaptive filtering, and EffiWeightNet for effective weight computation. Meanwhile, to maintain modality identity consistency, we propose the Causal Modality Identity Module (CMIM) that establishes causal constraints between generated images and target modality descriptions using vision-language modeling. Extensive experimental results demonstrate that our proposed Med-K2N outperforms state-of-the-art methods by significant margins on multiple benchmarks. Source code is available.

Figures (8)

• Figure 1: The overall framework of the proposed Med-K2N, it achieves flexible K-to-N modality mapping through a progressive fusion strategy of "key frame baseline + auxiliary modality step-by-step enhancement", addressing modality-differentiated modeling and identity confusion issues in traditional methods.
• Figure 2: Architecture of TaskHeadNet, illustrating the concurrent multi-head generation, quality-driven selection, and dynamic feedback mechanisms.
• Figure 3: Architecture of the Causal Modality Identity Module (CMIM). This module establishes causal consistency constraints between modality description texts and generated images through CLIP dual encoders, utilizing contrastive loss and metric learning to prevent modality identity confusion.
• Figure 4: Synthesis results on the Combined Brain Tumor Dataset. The figure demonstrates generation performance across four target modalities: T1c, T1n, T2f, and T2w.
• Figure 5: Representative qualitative visual comparisons of T1c images synthesized from T1n, T2w, and T2f sequences on the Combined Brain Tumor Dataset. Orange boxes highlight key reconstructed regions, with corresponding zoomed-in views provided. Yellow values indicate PSNR scores.