Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Cascaded Diffusion Models for 2D and 3D Microscopy Image Synthesis to Enhance Cell Segmentation

Rüveyda Yilmaz, Kaan Keven, Yuli Wu, Johannes Stegmaier

TL;DR

This work proposes a novel framework for synthesizing densely annotated 2D and 3D cell microscopy images using cascaded diffusion models and shows that training a segmentation model with a combination of synthetic data and real data improves cell segmentation performance by up to 9% across multiple datasets.

Abstract

Automated cell segmentation in microscopy images is essential for biomedical research, yet conventional methods are labor-intensive and prone to error. While deep learning-based approaches have proven effective, they often require large annotated datasets, which are scarce due to the challenges of manual annotation. To overcome this, we propose a novel framework for synthesizing densely annotated 2D and 3D cell microscopy images using cascaded diffusion models. Our method synthesizes 2D and 3D cell masks from sparse 2D annotations using multi-level diffusion models and NeuS, a 3D surface reconstruction approach. Following that, a pretrained 2D Stable Diffusion model is finetuned to generate realistic cell textures and the final outputs are combined to form cell populations. We show that training a segmentation model with a combination of our synthetic data and real data improves cell segmentation performance by up to 9\% across multiple datasets. Additionally, the FID scores indicate that the synthetic data closely resembles real data. The code for our proposed approach will be available at https://github.com/ruveydayilmaz0/cascaded_diffusion.

Cascaded Diffusion Models for 2D and 3D Microscopy Image Synthesis to Enhance Cell Segmentation

TL;DR

This work proposes a novel framework for synthesizing densely annotated 2D and 3D cell microscopy images using cascaded diffusion models and shows that training a segmentation model with a combination of synthetic data and real data improves cell segmentation performance by up to 9% across multiple datasets.

Abstract

Automated cell segmentation in microscopy images is essential for biomedical research, yet conventional methods are labor-intensive and prone to error. While deep learning-based approaches have proven effective, they often require large annotated datasets, which are scarce due to the challenges of manual annotation. To overcome this, we propose a novel framework for synthesizing densely annotated 2D and 3D cell microscopy images using cascaded diffusion models. Our method synthesizes 2D and 3D cell masks from sparse 2D annotations using multi-level diffusion models and NeuS, a 3D surface reconstruction approach. Following that, a pretrained 2D Stable Diffusion model is finetuned to generate realistic cell textures and the final outputs are combined to form cell populations. We show that training a segmentation model with a combination of our synthetic data and real data improves cell segmentation performance by up to 9\% across multiple datasets. Additionally, the FID scores indicate that the synthetic data closely resembles real data. The code for our proposed approach will be available at https://github.com/ruveydayilmaz0/cascaded_diffusion.

Paper Structure

This paper contains 6 sections, 2 equations, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Methods
  3. Experiments
  4. Conclusion
  5. COMPLIANCE WITH ETHICAL STANDARDS
  6. Acknowledgments

Figures (2)

  • Figure 1: Overview of the proposed method. For 2D data synthesis, MaskDDPM () and Stable Diffusion () generate masks and cell textures respectively. For 3D data generation, SyncDreamer, NeuS and volume slicing () are additionally employed. The final images and the masks are combined using the population synthesis module () .
  • Figure 2: Representative examples from the real and synthetic (a) HeLa, (b) GOWT1, (c) CE, and (d) CHO datasets. For the 3D CE and CHO datasets, zoomed-in 2D cross-sectional images along the X and Y dimensions are also provided.