PMPBench: A Paired Multi-Modal Pan-Cancer Benchmark for Medical Image Synthesis

TL;DR

PMPBench introduces the first public, fully paired pan-cancer CE/NCE imaging dataset spanning 11 organs, enabling systematic 1→1, N→1, and N→N translation benchmarks across MR and CT. The authors propose FlowMI, a flow-based latent-space imputation method using a product-of-experts VAE to handle arbitrary missing modalities, achieving superior reconstruction fidelity over GAN, diffusion, and direct translation baselines. The benchmark demonstrates FlowMI’s ability to recover clinically relevant contrast enhancement (e.g., DCE phases) with improved structural fidelity, supporting safer, efficient multi-organ oncology imaging workflows. This work provides a scalable resource and a robust method for evaluating and advancing multimodal image synthesis in oncology, with direct implications for diagnostic accuracy and workflow optimization.

Abstract

Contrast medium plays a pivotal role in radiological imaging, as it amplifies lesion conspicuity and improves detection for the diagnosis of tumor-related diseases. However, depending on the patient's health condition or the medical resources available, the use of contrast medium is not always feasible. Recent work has explored AI-based image translation to synthesize contrast-enhanced images directly from non-contrast scans, aims to reduce side effects and streamlines clinical workflows. Progress in this direction has been constrained by data limitations: (1) existing public datasets focus almost exclusively on brain-related paired MR modalities; (2) other collections include partially paired data but suffer from missing modalities/timestamps and imperfect spatial alignment; (3) explicit labeling of CT vs. CTC or DCE phases is often absent; (4) substantial resources remain private. To bridge this gap, we introduce the first public, fully paired, pan-cancer medical imaging dataset spanning 11 human organs. The MR data include complete dynamic contrast-enhanced (DCE) sequences covering all three phases (DCE1-DCE3), while the CT data provide paired non-contrast and contrast-enhanced acquisitions (CTC). The dataset is curated for anatomical correspondence, enabling rigorous evaluation of 1-to-1, N-to-1, and N-to-N translation settings (e.g., predicting DCE phases from non-contrast inputs). Built upon this resource, we establish a comprehensive benchmark. We report results from representative baselines of contemporary image-to-image translation. We release the dataset and benchmark to catalyze research on safe, effective contrast synthesis, with direct relevance to multi-organ oncology imaging workflows. Our code and dataset are publicly available at https://github.com/YifanChen02/PMPBench.

Figures (4)

• Figure 1: Top: representative organ-system with paired contrast and non-contrast scans. The charts imply organ-wise composition and modality balance. Bottom: standardized curation pipeline including data collection, cleaning, pairing, and registration.
• Figure 2: Representative task settings with examples. (a) CT $\rightarrow$ CTC (1$\rightarrow$1), (b) DCE$_1$$\rightarrow$ DCE$2$ (1$\rightarrow$1), (c) DCE${1,3}$$\rightarrow$ DCE$_2$ (N$\rightarrow$1), (d) DCE$1$$\rightarrow$ DCE${2,3}$ (1$\rightarrow$N).
• Figure 3: Overview of the proposed FlowMI framework. Left: Modality-specific encoders $E_\theta^{(i)}$ map inputs into a latent space, which are fused via a product-of-experts. The distribution with all modalities defines the target, while cases with missing modalities define the broken distribution. Middle: Latent flow matching learns a smooth mapping from $p_0(z)$ (broken) to $p_1(z)$ (target) using a U-Net parameterization of the velocity field, $\mathbf{u}_t = \tfrac{dz_t}{dt}$. Right: During inference, inputs with missing modalities are encoded and aligned through the learned flow, enabling consistent reconstruction or synthesis of complete modalities.
• Figure 4: CT$\rightarrow$CTC (liver). Red circles mark tumor regions. Input CT shows no clear lesion, ground-truth CTC shows bright enhancement. Most methods under-enhance the tumor, while ours recover the correct signal. Alongside each result, the blue residual maps visualize differences from ground truth (darker indicates larger error).