Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Denoising Diffusion Models for 3D Healthy Brain Tissue Inpainting

Alicia Durrer, Julia Wolleb, Florentin Bieder, Paul Friedrich, Lester Melie-Garcia, Mario Ocampo-Pineda, Cosmin I. Bercea, Ibrahim E. Hamamci, Benedikt Wiestler, Marie Piraud, Özgür Yaldizli, Cristina Granziera, Bjoern H. Menze, Philippe C. Cattin, Florian Kofler

TL;DR

This work modify state-of-the-art 2D, pseudo-3D, and 3D methods working in the image space, as well as 3D latent and 3D wavelet diffusion models, and train them to synthesize healthy brain tissue and shows that the pseudo-3D model performs best regarding the structural-similarity index, peak signal-to-noise ratio, and mean squared error.

Abstract

Monitoring diseases that affect the brain's structural integrity requires automated analysis of magnetic resonance (MR) images, e.g., for the evaluation of volumetric changes. However, many of the evaluation tools are optimized for analyzing healthy tissue. To enable the evaluation of scans containing pathological tissue, it is therefore required to restore healthy tissue in the pathological areas. In this work, we explore and extend denoising diffusion models for consistent inpainting of healthy 3D brain tissue. We modify state-of-the-art 2D, pseudo-3D, and 3D methods working in the image space, as well as 3D latent and 3D wavelet diffusion models, and train them to synthesize healthy brain tissue. Our evaluation shows that the pseudo-3D model performs best regarding the structural-similarity index, peak signal-to-noise ratio, and mean squared error. To emphasize the clinical relevance, we fine-tune this model on data containing synthetic MS lesions and evaluate it on a downstream brain tissue segmentation task, whereby it outperforms the established FMRIB Software Library (FSL) lesion-filling method.

Denoising Diffusion Models for 3D Healthy Brain Tissue Inpainting

TL;DR

This work modify state-of-the-art 2D, pseudo-3D, and 3D methods working in the image space, as well as 3D latent and 3D wavelet diffusion models, and train them to synthesize healthy brain tissue and shows that the pseudo-3D model performs best regarding the structural-similarity index, peak signal-to-noise ratio, and mean squared error.

Abstract

Monitoring diseases that affect the brain's structural integrity requires automated analysis of magnetic resonance (MR) images, e.g., for the evaluation of volumetric changes. However, many of the evaluation tools are optimized for analyzing healthy tissue. To enable the evaluation of scans containing pathological tissue, it is therefore required to restore healthy tissue in the pathological areas. In this work, we explore and extend denoising diffusion models for consistent inpainting of healthy 3D brain tissue. We modify state-of-the-art 2D, pseudo-3D, and 3D methods working in the image space, as well as 3D latent and 3D wavelet diffusion models, and train them to synthesize healthy brain tissue. Our evaluation shows that the pseudo-3D model performs best regarding the structural-similarity index, peak signal-to-noise ratio, and mean squared error. To emphasize the clinical relevance, we fine-tune this model on data containing synthetic MS lesions and evaluate it on a downstream brain tissue segmentation task, whereby it outperforms the established FMRIB Software Library (FSL) lesion-filling method.
Paper Structure (13 sections, 4 equations, 3 figures, 3 tables)

This paper contains 13 sections, 4 equations, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Contribution
  4. Methods
  5. Denoising Diffusion Probabilistic Models
  6. Modifying Diffusion Models for Inpainting
  7. Experiments
  8. Datasets
  9. Implementation Details
  10. Evaluation Metrics
  11. Results
  12. Discussion
  13. Conclusion

Figures (3)

  • Figure 1: Automatic MR image processing tools are often designed to evaluate healthy tissue only. Therefore, pathological tissue needs to be replaced by healthy tissue. We modify and evaluate 2D, pseudo-3D and 3D denoising diffusion models to obtain consistent 3D healthy tissue inpainting.
  • Figure 2: We present an overview of the denoising process adapted for the inpainting task. For conditioning, the mask $m$ and the masked image $b$ are concatenated to the noisy image $x_t$, resulting in $X_t$, serving as input for the diffusion model. The output of the diffusion model is the predicted noise $\epsilon_{\theta}(X_t,t)$.
  • Figure 3: Left: Qualitative comparison of DDPM 2D slice-wise and DDPM Pseudo3D on an example of the BraTS test set. Right: Magnified view of the inpainting by FSL lesion filling and DDPM Pseudo3D on an example of the MS test set. The blue box and the blue arrows indicate image artifacts.