Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MRPD: Undersampled MRI reconstruction by prompting a large latent diffusion model

Ziqi Gao, S. Kevin Zhou

TL;DR

MRPD tackles undersampled MRI reconstruction by prompting a large latent diffusion model (LLDM) pre-trained on natural images. It introduces MRSampler for unsupervised, latent-domain guidance with hard-to-soft consistency and random phase modulation, and MRAdapter for universal fine-tuning of the LLDM's autoencoder using minimal parameters. Across FastMRI and IXI, MRPD demonstrates database-free and database-available capabilities with strong generalization to out-of-domain samplings, contrasts, and organs, often outperforming supervised and diffusion-based baselines. The framework highlights the universal visual knowledge embedded in LLDMs and presents a practical path toward generalizable, foundation-model-driven MRI reconstruction, with potential for acceleration and broader applicability.

Abstract

Implicit visual knowledge in a large latent diffusion model (LLDM) pre-trained on natural images is rich and hypothetically universal to natural and medical images. To test this hypothesis from a practical perspective, we propose a novel framework for undersampled MRI Reconstruction by Prompting a large latent Diffusion model (MRPD). While the existing methods trained on MRI datasets are typically of limited generalizability toward diverse data acquisition scenarios, MRPD supports unsupervised and universally adaptive MRI reconstruction. For unsupervised reconstruction, MRSampler guides LLDM with a random-phase-modulated hard-to-soft control. With any single- or multiple-source MRI dataset, MRPD's performance is boosted universally by a lightweight MRAdapter that only finetunes the LLDM's autoencoder. Experiments on FastMRI and IXI show that MRPD is the only model that supports both MRI database-free and database-available scenarios and attains the best generalizability towards out-of-domain (OOD) samplings, contrasts, and organs among compared unsupervised, supervised, and MRI diffusion methods. To our knowledge, MRPD is the first method that empirically shows the universal prowess of an LLDM pre-trained on vast natural images for MRI. Our official implementation is at https://github.com/Z7Gao/MRPD.

MRPD: Undersampled MRI reconstruction by prompting a large latent diffusion model

TL;DR

MRPD tackles undersampled MRI reconstruction by prompting a large latent diffusion model (LLDM) pre-trained on natural images. It introduces MRSampler for unsupervised, latent-domain guidance with hard-to-soft consistency and random phase modulation, and MRAdapter for universal fine-tuning of the LLDM's autoencoder using minimal parameters. Across FastMRI and IXI, MRPD demonstrates database-free and database-available capabilities with strong generalization to out-of-domain samplings, contrasts, and organs, often outperforming supervised and diffusion-based baselines. The framework highlights the universal visual knowledge embedded in LLDMs and presents a practical path toward generalizable, foundation-model-driven MRI reconstruction, with potential for acceleration and broader applicability.

Abstract

Implicit visual knowledge in a large latent diffusion model (LLDM) pre-trained on natural images is rich and hypothetically universal to natural and medical images. To test this hypothesis from a practical perspective, we propose a novel framework for undersampled MRI Reconstruction by Prompting a large latent Diffusion model (MRPD). While the existing methods trained on MRI datasets are typically of limited generalizability toward diverse data acquisition scenarios, MRPD supports unsupervised and universally adaptive MRI reconstruction. For unsupervised reconstruction, MRSampler guides LLDM with a random-phase-modulated hard-to-soft control. With any single- or multiple-source MRI dataset, MRPD's performance is boosted universally by a lightweight MRAdapter that only finetunes the LLDM's autoencoder. Experiments on FastMRI and IXI show that MRPD is the only model that supports both MRI database-free and database-available scenarios and attains the best generalizability towards out-of-domain (OOD) samplings, contrasts, and organs among compared unsupervised, supervised, and MRI diffusion methods. To our knowledge, MRPD is the first method that empirically shows the universal prowess of an LLDM pre-trained on vast natural images for MRI. Our official implementation is at https://github.com/Z7Gao/MRPD.
Paper Structure (36 sections, 4 equations, 5 figures, 9 tables, 1 algorithm)

This paper contains 36 sections, 4 equations, 5 figures, 9 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. Diffusion Models
  4. Diffusion Models for MRI Reconstruction
  5. Method
  6. Problem formulation
  7. Unsupervised MRSampler
  8. Hard-to-Soft Diffusion Guidance
  9. Random Phase Modulation
  10. Universally Adaptive MRAdapter
  11. MRPD meets Parallel Imaging (PI)
  12. Experiments
  13. Experimental Settings
  14. Datasets
  15. Baselines
  16. ...and 21 more sections

Figures (5)

  • Figure 1: Overview of MRPD. (a) Unconditional image generation of LLDM with a deterministic DDIM sampler. (b) Image-specific undersampled MRI reconstruction with an MRSampler. (c) MRAdapter for MRI database-available scenarios and the composition of an LLDM, Stable Diffusion v1.5, with our adaptation.
  • Figure 2: RPM's effect. Panel (a) shows an intermediate clean image $\hat{x}_0$ guided by hard DC without and with RPM and (b) shows the impact of RPM on the k-space domain with the corresponding images.
  • Figure 3: Visualization of an in-D PD-w knee image undersampled with diverse masks. A sub-figure has a reconstructed image and its error map.
  • Figure 4: Pareto Frontier of inference time and PSNR under different combinations of $t_0$ and $t_{ws}$. The $\times$ markers represent feasible choices within the range of $t_0$,$t_{ws}$. The black line is the Pareto-efficient frontier and the gray line is a time budget of 50s. Our chosen value is circled.
  • Figure 5: Image reconstruction evolution during the diffusion process guided by the MRSampler and other unitary or magnitude-only controllers. (a) The evolution of PSNR and SSIM. (b)-(d) In each panel, the first row illustrates the evolution of clean image predictions $\hat{x}_0(z_t)$, and the second row depicts the intermediate noisy images $z_t$.