Slice-Consistent 3D Volumetric Brain CT-to-MRI Translation with 2D Brownian Bridge Diffusion Model

TL;DR

This work tackles CT-to-MRI translation by addressing diffusion-model stochasticity and slice inconsistency in 3D volumes. It introduces Style Key Conditioning (SKC) to control MRI style via histogram-based conditioning and Inter-Slice Trajectory Alignment (ISTA) to synchronize adjacent slices during sampling, all within a 2D Brownian Bridge diffusion framework. The approach yields fully deterministic, slice-consistent 3D brain volumes and outperforms 2D and 3D baselines on in-house CT-MRI and BraTS-FLAIR-T1 data, using metrics such as $NRMSE$, $PSNR$, and $SSIM$. This method has practical implications for cost-effective, reliable multimodal imaging and could extend to MRI harmonization and broader medical volume synthesis tasks.

Abstract

In neuroimaging, generally, brain CT is more cost-effective and accessible imaging option compared to MRI. Nevertheless, CT exhibits inferior soft-tissue contrast and higher noise levels, yielding less precise structural clarity. In response, leveraging more readily available CT to construct its counterpart MRI, namely, medical image-to-image translation (I2I), serves as a promising solution. Particularly, while diffusion models (DMs) have recently risen as a powerhouse, they also come with a few practical caveats for medical I2I. First, DMs' inherent stochasticity from random noise sampling cannot guarantee consistent MRI generation that faithfully reflects its CT. Second, for 3D volumetric images which are prevalent in medical imaging, naively using 2D DMs leads to slice inconsistency, e.g., abnormal structural and brightness changes. While 3D DMs do exist, significant training costs and data dependency bring hesitation. As a solution, we propose novel style key conditioning (SKC) and inter-slice trajectory alignment (ISTA) sampling for the 2D Brownian bridge diffusion model. Specifically, SKC ensures a consistent imaging style (e.g., contrast) across slices, and ISTA interconnects the independent sampling of each slice, deterministically achieving style and shape consistent 3D CT-to-MRI translation. To the best of our knowledge, this study is the first to achieve high-quality 3D medical I2I based only on a 2D DM with no extra architectural models. Our experimental results show superior 3D medical I2I than existing 2D and 3D baselines, using in-house CT-MRI dataset and BraTS2023 FLAIR-T1 MRI dataset.

Figures (7)

• Figure 1: Examples of slice inconsistency and resolved outcomes. This figure displays the coronal view of volumes synthesized by two 2D BBDMs trained on axial slices. The pure multi-slice 2D BBDM exhibits severe slice inconsistency, with noticeable discontinuities in both style and shape across slices. Our method produces slice-consistent volumes and can adjust the intensity histogram (i.e., style).
• Figure 2: Training and sampling scheme of the proposed methods.(a) During the multi-slice BBDM training, a target histogram-based style key is injected into the U-Net. (b) Target volume sampling proceeds in the manner of the Predictor-Corrector method. During the co-prediction phase, multiple $\bm{\epsilon}^{i,k}_{\theta,t}$ are employed to establish connections among the predicted slices within $\bm{\bar{X}}_{t-1}$. In the subsequent correction phase, the co-predicted volume is refined through a score-guided deterministic process.
• Figure 3: Visualization of the latent space and algorithm for ISTA sampling. The trained U-Net produces inconsistent outputs for multi-slice inputs that include the $i^{th}$ slice. The co-prediction unifies the direction of these independent inferences, while the correction aligns the co-predicted $\bar{\bm{x}}_{t}^i$ onto the manifold of $\bm{x}^i_{t}$.
• Figure 4: Qualitative comparison with baselines (CT$\rightarrow$MRI)
• Figure 5: Qualitative comparison with baselines (FLAIR$\rightarrow$T1)