Regression is all you need for medical image translation

TL;DR

This paper investigates whether diffusion-based medical image translation (MIT) truly outperforms simpler regression approaches. It introduces YODA, a 2.5D diffusion framework that can operate with regression sampling to produce noise-free translations in a single step, and ExpA sampling to approximate the diffusion model's expectation via averaging multiple samples. Across five diverse datasets (including RS, MBB, BraTS, IXI, and Gold Atlas MRI->CT), YODA consistently matches or outperforms state-of-the-art GANs and diffusion models in medically relevant metrics, while diffusion sampling often artificially increases perceptual realism by replicating acquisition noise. The key finding is that iterative diffusion refinement primarily adds noise rather than new medical information, and regression sampling yields superior fidelity for downstream clinical tasks, suggesting fidelity-focused baselines should be the standard for MIT evaluation. YODA's approach also demonstrates potential for producing synthetic images interchangeable with acquisitions in several medical applications, with strong generalization to unseen data sources.

Abstract

While Generative Adversarial Nets (GANs) and Diffusion Models (DMs) have achieved impressive results in natural image synthesis, their core strengths - creativity and realism - can be detrimental in medical applications, where accuracy and fidelity are paramount. These models instead risk introducing hallucinations and replication of unwanted acquisition noise. Here, we propose YODA (You Only Denoise once - or Average), a 2.5D diffusion-based framework for medical image translation (MIT). Consistent with DM theory, we find that conventional diffusion sampling stochastically replicates noise. To mitigate this, we draw and average multiple samples, akin to physical signal averaging. As this effectively approximates the DM's expected value, we term this Expectation-Approximation (ExpA) sampling. We additionally propose regression sampling YODA, which retains the initial DM prediction and omits iterative refinement to produce noise-free images in a single step. Across five diverse multi-modal datasets - including multi-contrast brain MRI and pelvic MRI-CT - we demonstrate that regression sampling is not only substantially more efficient but also matches or exceeds image quality of full diffusion sampling even with ExpA. Our results reveal that iterative refinement solely enhances perceptual realism without benefiting information translation, which we confirm in relevant downstream tasks. YODA outperforms eight state-of-the-art DMs and GANs and challenges the presumed superiority of DMs and GANs over computationally cheap regression models for high-quality MIT. Furthermore, we show that YODA-translated images are interchangeable with, or even superior to, physical acquisitions for several medical applications.

Figures (11)

• Figure 1: Inherent differences of Medical Image Translation (MIT) and Natural Image Generation (NIG) and the consequence for DDPM sampling. (a,b) Exemplary MIT and NIG image density functions are projected for visualization. (a) The noisy image acquisition and the strong condition required for faithful MIT (e.g., T1w,T2w $\rightarrow$ FLAIR) creates a uni-modal cluster of acquirable noisy images $X$ centered around the noise-free image $X'$. (b) Conversely, the complex image formation and typically weak condition in NIG (e.g., CIFAR-10 cifar class conditional) leads to a density landscape with scattered and distinctive modes (compare also Fig. 3 in ledig2017srgan). (c + d) Sampling trajectories and exemplary outputs $\hat{X}_{T\rightarrow0}$ of ideal MIT (c) and NIG (d) denoisers $p_\theta$ (see also ho2020denoisingkarras2022elucidating). (c) Note that the generated MIT samples $\hat{X}_{1\rightarrow 0}^{(j)}$ share the medical information (e.g., lesions) but differ in the noise manifestations (best viewed zoomed in). Thus, image averages are valid ($p>0$) images. (d) Conversely, NIG sampling branches into distinctive objects of the class (e.g., "airplane") ho2020denoisingkarras2022elucidating, such that averages are predominantly invalid ($p=0$) images. (e)Proposed Expectation-Approximation (ExpA) Sampling (Samp.): Similar to physical signal averages, uncorrelated noise in samples $\hat{X}_{1\rightarrow0}^{(j)}, \; j \in [1:N_\text{Ex}]$ drawn from $p_\theta (C)$ enables noise suppression by averaging $X_\text{ExpA}=\text{avg}_j(\hat{X}_{1\rightarrow 0}^{(j)})$. For an ideal $p_\theta(C)$, averaging many samples slowly approaches $\mathbb{E}_{X \sim \mathcal{D}_{C}}\left[X\right]$ (see also the supplementary video). (f)Proposed Regression Sampling (Regr.): We simply use the initial minimum-mean-squared-error (MMSE) solution, $\hat{X}_{T\rightarrow0}$, without further refinement and, thus, noise generation to directly approximate $\mathbb{E}_{X \sim \mathcal{D}_{C}}\left[X\right]$. (g) Whereas ExpA requires full DDPM sampling and exacerbates the intrinsically high number of function evaluations ($N_\text{FE}$) of DDPMs, regression sampling leverages the single-step MMSE solution requiring only $N_\text{FE}=1$. Note that ExpA and regression sampling both approximate the superposition of all valid objects given $C$. While in NIG, this is meaningless ledig2017srgankarras2022elucidating, the strong conditioning and, thus, semantic determinism of MIT make this superposition well-defined with a correspondence to the noise-free image $X'$.
• Figure 2: 2.5D diffusion sampling of YODA: In each time step $t$, the latent diffusion volume $X_t \in \mathbb{R}^{D\times H \times W}$ is processed in slabs of $n$ consecutive slices (default: $n=5$). The slabs are concatenated with the conditioning $C$, i.e. the corresponding slabs of the source images as input the neural denoiser $v_\theta$. The output slices are then stacked as predicted volume $\hat{X}_{t\rightarrow0}$ to which noise $\varepsilon$ is added to obtain the next latent image $\hat{X}_{t-1}$. Note, that the slicing plane is rotated between axial, coronal, and sagittal.
• Figure 3: In regular DM sampling (green), the diffusion prior $\hat{X} _t$ for time steps $t {\in}[1k{:}250]$ is weak compared to the conditioning $C$. Thus, the estimate of the noise-free images ($\hat{X}_{t\rightarrow0}$, line 3 of Alg. \ref{['algo:native_sample']}) remains almost unchanged (best view zoomed-in). We therefore truncate DM sampling skipping the time steps between $t{=}999$ and $250$ (red arrow).
• Figure 4: Diffusion, ExpA, and regression sampling of YODA is demonstrated against competing methods for T1w,T2w $\rightarrow$ FLAIR translation on random images of the RS (upper) and BraTS (lower). Note that all sampling methods of YODA allow for a more faithful translation of lesions while avoiding artifacts such as Salt-and-Pepper noise and other unrealistic textures. See also the supplementary video for additional translation results.
• Figure 5: FLAIR images generated by YODA with diffusion (diff.), ExpA ($N_\text{Ex}{=}10$), and regression (regr.) sampling [...] are compared to an acquired image (see also the supplementary video). Note that ExpA resembles regression sampling and achieves crisper edges but looses fine-grained details compared to diffusion sampling. Additionally, the smooth regression image is noised with Rician noise (resulting std.: 1.4% of $\operatorname{range}(\hat{X})$) to restore perceptual realism.