Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Improving 2D Diffusion Models for 3D Medical Imaging with Inter-Slice Consistent Stochasticity

Chenhe Du, Qing Wu, Xuanyu Tian, Jingyi Yu, Hongjiang Wei, Yuyao Zhang

TL;DR

This work tackles the challenge of leveraging 2D diffusion priors for 3D medical imaging, where independent slice-wise sampling induces inter-slice discontinuities. The authors diagnose the root cause as uncoordinated stochasticity during the reverse diffusion process and propose Inter-Slice Consistent Stochasticity (ISCS), a plug-and-play method that generates correlated per-slice noise via spherical linear interpolation (Slerp) on the noise hypersphere, without retraining or extra losses. ISCS is integrated into existing diffusion-based solvers by replacing standard noise in the re-noising step with ε^{ISCS}, aligning sampling trajectories across slices. Empirical results on sparse-view CT, limited-angle CT, and MRI isotropic SR show consistent improvements over 2D priors and competitive performance against 3D-aware priors, with preserved edge detail and reduced artifacts. Overall, the approach offers a practical route to high-fidelity 3D medical imaging using 2D diffusion priors, by modulating the stochasticity during sampling rather than modifying models or training regimes.

Abstract

3D medical imaging is in high demand and essential for clinical diagnosis and scientific research. Currently, diffusion models (DMs) have become an effective tool for medical imaging reconstruction thanks to their ability to learn rich, high-quality data priors. However, learning the 3D data distribution with DMs in medical imaging is challenging, not only due to the difficulties in data collection but also because of the significant computational burden during model training. A common compromise is to train the DMs on 2D data priors and reconstruct stacked 2D slices to address 3D medical inverse problems. However, the intrinsic randomness of diffusion sampling causes severe inter-slice discontinuities of reconstructed 3D volumes. Existing methods often enforce continuity regularizations along the z-axis, which introduces sensitive hyper-parameters and may lead to over-smoothing results. In this work, we revisit the origin of stochasticity in diffusion sampling and introduce Inter-Slice Consistent Stochasticity (ISCS), a simple yet effective strategy that encourages interslice consistency during diffusion sampling. Our key idea is to control the consistency of stochastic noise components during diffusion sampling, thereby aligning their sampling trajectories without adding any new loss terms or optimization steps. Importantly, the proposed ISCS is plug-and-play and can be dropped into any 2D trained diffusion based 3D reconstruction pipeline without additional computational cost. Experiments on several medical imaging problems show that our method can effectively improve the performance of medical 3D imaging problems based on 2D diffusion models. Our findings suggest that controlling inter-slice stochasticity is a principled and practically attractive route toward high-fidelity 3D medical imaging with 2D diffusion priors. The code is available at: https://github.com/duchenhe/ISCS

Improving 2D Diffusion Models for 3D Medical Imaging with Inter-Slice Consistent Stochasticity

TL;DR

This work tackles the challenge of leveraging 2D diffusion priors for 3D medical imaging, where independent slice-wise sampling induces inter-slice discontinuities. The authors diagnose the root cause as uncoordinated stochasticity during the reverse diffusion process and propose Inter-Slice Consistent Stochasticity (ISCS), a plug-and-play method that generates correlated per-slice noise via spherical linear interpolation (Slerp) on the noise hypersphere, without retraining or extra losses. ISCS is integrated into existing diffusion-based solvers by replacing standard noise in the re-noising step with ε^{ISCS}, aligning sampling trajectories across slices. Empirical results on sparse-view CT, limited-angle CT, and MRI isotropic SR show consistent improvements over 2D priors and competitive performance against 3D-aware priors, with preserved edge detail and reduced artifacts. Overall, the approach offers a practical route to high-fidelity 3D medical imaging using 2D diffusion priors, by modulating the stochasticity during sampling rather than modifying models or training regimes.

Abstract

3D medical imaging is in high demand and essential for clinical diagnosis and scientific research. Currently, diffusion models (DMs) have become an effective tool for medical imaging reconstruction thanks to their ability to learn rich, high-quality data priors. However, learning the 3D data distribution with DMs in medical imaging is challenging, not only due to the difficulties in data collection but also because of the significant computational burden during model training. A common compromise is to train the DMs on 2D data priors and reconstruct stacked 2D slices to address 3D medical inverse problems. However, the intrinsic randomness of diffusion sampling causes severe inter-slice discontinuities of reconstructed 3D volumes. Existing methods often enforce continuity regularizations along the z-axis, which introduces sensitive hyper-parameters and may lead to over-smoothing results. In this work, we revisit the origin of stochasticity in diffusion sampling and introduce Inter-Slice Consistent Stochasticity (ISCS), a simple yet effective strategy that encourages interslice consistency during diffusion sampling. Our key idea is to control the consistency of stochastic noise components during diffusion sampling, thereby aligning their sampling trajectories without adding any new loss terms or optimization steps. Importantly, the proposed ISCS is plug-and-play and can be dropped into any 2D trained diffusion based 3D reconstruction pipeline without additional computational cost. Experiments on several medical imaging problems show that our method can effectively improve the performance of medical 3D imaging problems based on 2D diffusion models. Our findings suggest that controlling inter-slice stochasticity is a principled and practically attractive route toward high-fidelity 3D medical imaging with 2D diffusion priors. The code is available at: https://github.com/duchenhe/ISCS
Paper Structure (51 sections, 1 theorem, 19 equations, 9 figures, 9 tables, 1 algorithm)

This paper contains 51 sections, 1 theorem, 19 equations, 9 figures, 9 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Diffusion Models
  4. Medical Imaging Inverse Problem
  5. DM-based Medical Inverse Problem Solvers
  6. Proposed Method
  7. Inter-Slice Inconsistency in 2D DM-based 3D Reconstruction
  8. Slice-wise Approximation of 3D Diffusion Prior
  9. Root Cause of Inter-Slice Inconsistency
  10. Limitations of Post-Hoc Regularization
  11. Our Motivation
  12. Inter-Slice Consistent Stochasticity
  13. Batch-Consistent Sampling & Limitations
  14. Correlated Noise Generation via Slerp
  15. Integration with Diffusion Samplers
  16. ...and 36 more sections

Key Result

Proposition B.1

Let $z_1, z_2 \sim \mathcal{N}(0, I_d)$ be two independent random vectors in $\mathbb{R}^d$. As the dimension $d \to \infty$, the angle $\theta(z_1, z_2)$ between them converges in probability to $\pi/2$:

Figures (9)

  • Figure 1: Geometric interpretation of how different noise strategies in the re-noising step affect the stochasticity and resulting consistency in diffusion sampling. (a) Independent Noise (Conventional): Independently sampled noise for each slice, leading to uncorrelated sampling paths. (b) Identical Noise (BCS kwon2025solving): Applying the same noise to all slices forces identical sampling paths. (c) Slerp Noise (Ours): Our proposed ISCS interpolates noise on the hypersphere, generating smoothly correlated information across slices.
  • Figure 2: Qualitative results of compared methods on a representative sample for SVCT of 30 views. The display window is set as [-480, 820] HU.
  • Figure 3: Qualitative results of compared methods on a representative sample for MRI SR of 5$\times$.
  • Figure 4: Qualitative results of adopting identical (BCS) and slerp noise (ISCS) during re-noising, where the red arrows denote the noticeable "copying artifacts". The display window is set as [-480, 820] HU.
  • Figure 5: Performance curves across the sampling process, where a higher PSNR and lower LPIPS and inter-slice difference reflect improved data fidelity and better inter-slice consistency.
  • ...and 4 more figures

Theorems & Definitions (2)

  • Proposition B.1: Concentration of Angular Distance in High Dimensions
  • Remark B.1