X-Diffusion: Generating Detailed 3D MRI Volumes From a Single Image Using Cross-Sectional Diffusion Models

TL;DR

X-Diffusion presents a cross-sectional diffusion framework that can generate detailed 3D MRI volumes from extremely sparse 2D inputs by learning across cross-sectional views and aggregating view-conditioned volumes. It treats MRI data as holistic 3D volumes during both training and inference, enabling 2D-to-3D reconstruction from a single slice or a few slices and achieving state-of-the-art PSNR with minimal input, shown on BRATS brain and UK Biobank body MRIs and generalizing to knee MRIs. The method preserves critical anatomical features (tumor profiles, spine curvature, brain volume) and is validated by medical experts, who often cannot distinguish generated from real MRIs, suggesting clinical relevance. The approach leverages large-scale pretraining (Stable Diffusion LDM, Zero-123) and a multi-view volume averaging scheme to achieve robust, out-of-domain generalization, potentially enabling faster, cheaper MRI workflows in the future.

Abstract

Magnetic Resonance Imaging (MRI) is a crucial diagnostic tool, but high-resolution scans are often slow and expensive due to extensive data acquisition requirements. Traditional MRI reconstruction methods aim to expedite this process by filling in missing frequency components in the K-space, performing 3D-to-3D reconstructions that demand full 3D scans. In contrast, we introduce X-Diffusion, a novel cross-sectional diffusion model that reconstructs detailed 3D MRI volumes from extremely sparse spatial-domain inputs, achieving 2D-to-3D reconstruction from as little as a single 2D MRI slice or few slices. A key aspect of X-Diffusion is that it models MRI data as holistic 3D volumes during the cross-sectional training and inference, unlike previous learning approaches that treat MRI scans as collections of 2D slices in standard planes (coronal, axial, sagittal). We evaluated X-Diffusion on brain tumor MRIs from the BRATS dataset and full-body MRIs from the UK Biobank dataset. Our results demonstrate that X-Diffusion not only surpasses state-of-the-art methods in quantitative accuracy (PSNR) on unseen data but also preserves critical anatomical features such as tumor profiles, spine curvature, and brain volume. Remarkably, the model generalizes beyond the training domain, successfully reconstructing knee MRIs despite being trained exclusively on brain data. Medical expert evaluations further confirm the clinical relevance and fidelity of the generated images. To our knowledge, X-Diffusion is the first method capable of producing detailed 3D MRIs from highly limited 2D input data, potentially accelerating MRI acquisition and reducing associated costs. The code is available on the project website https://emmanuelleb985.github.io/XDiffusion/ .

Figures (24)

• Figure 1: X-Diffusion for Sparse MRI Reconstruction. (Right) We present X-Diffusion, a method that can generate detailed and dense MRI volumes from a single MRI slice or a few slices. X-Diffusion is the first method in medical imaging to generate detailed 3D MRIs from extremely sparse inputs, preserving key anatomical properties. (Left) MRI reconstruction traditionally involves retrieving high-frequency images from low-frequency full 3D MRI volumes (in the K-space).
• Figure 2: X-Diffusion Pipeline. A single or multi-slice input is fed into the Latent Diffusion U-Net conditioned on the target slice index $d$ and target rotation from 360$^{\circ}$ slicing. The 3D volume is reconstructed by vertical stacking of the slices from a fixed axis of rotation. The final volume $\mathcal{X}$ is obtained after averaging the $N$ realigned view-dependant volumes $R_{i}^{\intercal}\mathcal{X}_{R_{i}}$ from a set of predefined target rotations $R_{i}$.
• Figure 3: Test Time Brain Generation at Different Sampling Steps. For the input slice index of 107 (left), we show the ground-truth slice 90 (right) and the corresponding brain slice generated at different sampling steps $t$ in the denoising diffusion process of X-Diffusion trained on BRATS.
• Figure 4: Qualitative Results of full body 3D MRI Generation with X-Diffusion. We show a single MRI slice example, two corresponding ground-truth MRI slices (index 68 and 100), the corresponding generated MRI slice, and a difference map to qualitatively measure the error between generated and ground-truth MRI.
• Figure 5: Visualisations of 3D Brain Generation. For the input slice (slice index 76), we show examples of slices from generated 3D brain MRI volumes with varying slice index (top) and its ground-truth brain slices (bottom). We show the tumour profile segmentation map in all output and ground truth slices and show the 3D tumor in the generated MRI and ground truth MRI in the most right column. Red is used for non-enhancing and necrotic tumor core, green for the peritumoral edema, and blue for the enhancing tumor core.