Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ReBrain: Brain MRI Reconstruction from Sparse CT Slice via Retrieval-Augmented Diffusion

Junming Liu, Yifei Sun, Weihua Cheng, Yujin Kang, Yirong Chen, Ding Wang, Guosun Zeng

TL;DR

ReBrain tackles the problem of reconstructing high-fidelity MRI from sparsely sampled CT by combining a Brownian Bridge diffusion model with retrieval-augmented guidance. A brain knowledge base enables non-local, structure-aware references that conditioning through ControlNet guides the diffusion process, while a SLERP fallback ensures stability when retrieval is weak. The approach achieves state-of-the-art cross-modal reconstruction under sparse input on SynthRAD2023 and BraTS, with strong volumetric continuity and robust ablations confirming the contribution of each component. This framework offers a practical route to clinically feasible MRI reconstruction in settings with limited CT data, while incorporating ethical safeguards and uncertainty awareness.

Abstract

Magnetic Resonance Imaging (MRI) plays a crucial role in brain disease diagnosis, but it is not always feasible for certain patients due to physical or clinical constraints. Recent studies attempt to synthesize MRI from Computed Tomography (CT) scans; however, low-dose protocols often result in highly sparse CT volumes with poor through-plane resolution, making accurate reconstruction of the full brain MRI volume particularly challenging. To address this, we propose ReBrain, a retrieval-augmented diffusion framework for brain MRI reconstruction. Given any 3D CT scan with limited slices, we first employ a Brownian Bridge Diffusion Model (BBDM) to synthesize MRI slices along the 2D dimension. Simultaneously, we retrieve structurally and pathologically similar CT slices from a comprehensive prior database via a fine-tuned retrieval model. These retrieved slices are used as references, incorporated through a ControlNet branch to guide the generation of intermediate MRI slices and ensure structural continuity. We further account for rare retrieval failures when the database lacks suitable references and apply spherical linear interpolation to provide supplementary guidance. Extensive experiments on SynthRAD2023 and BraTS demonstrate that ReBrain achieves state-of-the-art performance in cross-modal reconstruction under sparse conditions.

ReBrain: Brain MRI Reconstruction from Sparse CT Slice via Retrieval-Augmented Diffusion

TL;DR

ReBrain tackles the problem of reconstructing high-fidelity MRI from sparsely sampled CT by combining a Brownian Bridge diffusion model with retrieval-augmented guidance. A brain knowledge base enables non-local, structure-aware references that conditioning through ControlNet guides the diffusion process, while a SLERP fallback ensures stability when retrieval is weak. The approach achieves state-of-the-art cross-modal reconstruction under sparse input on SynthRAD2023 and BraTS, with strong volumetric continuity and robust ablations confirming the contribution of each component. This framework offers a practical route to clinically feasible MRI reconstruction in settings with limited CT data, while incorporating ethical safeguards and uncertainty awareness.

Abstract

Magnetic Resonance Imaging (MRI) plays a crucial role in brain disease diagnosis, but it is not always feasible for certain patients due to physical or clinical constraints. Recent studies attempt to synthesize MRI from Computed Tomography (CT) scans; however, low-dose protocols often result in highly sparse CT volumes with poor through-plane resolution, making accurate reconstruction of the full brain MRI volume particularly challenging. To address this, we propose ReBrain, a retrieval-augmented diffusion framework for brain MRI reconstruction. Given any 3D CT scan with limited slices, we first employ a Brownian Bridge Diffusion Model (BBDM) to synthesize MRI slices along the 2D dimension. Simultaneously, we retrieve structurally and pathologically similar CT slices from a comprehensive prior database via a fine-tuned retrieval model. These retrieved slices are used as references, incorporated through a ControlNet branch to guide the generation of intermediate MRI slices and ensure structural continuity. We further account for rare retrieval failures when the database lacks suitable references and apply spherical linear interpolation to provide supplementary guidance. Extensive experiments on SynthRAD2023 and BraTS demonstrate that ReBrain achieves state-of-the-art performance in cross-modal reconstruction under sparse conditions.

Paper Structure

This paper contains 28 sections, 26 equations, 12 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Cross-Modal Medical Image Reconstruction
  4. Intra-Volume Slice Interpolation
  5. Diffusion Models in Medical Imaging
  6. Methods
  7. Brownian Bridge Diffusion Model
  8. Knowledge Retrieval
  9. Conditional Guidance in BBDM via ControlNet
  10. Latent Interpolation as a Fallback for Retrieval
  11. Experiments
  12. Experimental Setups
  13. Results
  14. Ablation Study
  15. Further Analysis
  16. ...and 13 more sections

Figures (12)

  • Figure 1: Sparse CT slices (low axial resolution) lead to low-density MRI reconstructions with reduced inter-slice continuity, in contrast to the dense structure of standard MRI volumes.
  • Figure 2: Overview of the ReBrain workflow. (a) We construct a brain knowledge base to enable effective retrieval. (b) We train a BBDM with optimized directional noise to better recover MRI from CT. (c) During inference, a fine-tuned retrieval model finds a similar reference slice, which is fed into the BBDM through a ControlNet branch to guide MRI reconstruction.
  • Figure 3: Illustration of the brain knowledge base structure.
  • Figure 4: Representative cross-modal reconstruction results on the BraTS dataset. More results are provided in Appendix D.
  • Figure 5: Comparison of generation results using interpolation and retrieval under varying similarity conditions on BraTS.
  • ...and 7 more figures

Theorems & Definitions (3)

  • proof
  • proof : Sketch
  • proof : Remarks