Diffusion-Based Synthetic Brightfield Microscopy Images for Enhanced Single Cell Detection
Mario de Jesus da Graca, Jörg Dahlkemper, Peer Stelldinger
TL;DR
This work tackles the data scarcity challenge in brightfield single-cell detection by evaluating unconditional diffusion-based synthesis as a data-augmentation strategy. It trains a U-Net–based diffusion model on CHO cell patches to generate realistic brightfield images and assesses the impact on state-of-the-art detectors (YOLOv8, YOLOv9, RT-DETR). The experiments show that replacing or adding synthetic images to real data can achieve comparable or improved detection performance, especially at lower IoU thresholds, and the expert survey indicates high perceptual realism. The findings suggest diffusion-based synthetic data as a practical means to alleviate annotation burdens and bolster robustness in microscopy image analysis, with public code and data resources provided.
Abstract
Accurate single cell detection in brightfield microscopy is crucial for biological research, yet data scarcity and annotation bottlenecks limit the progress of deep learning methods. We investigate the use of unconditional models to generate synthetic brightfield microscopy images and evaluate their impact on object detection performance. A U-Net based diffusion model was trained and used to create datasets with varying ratios of synthetic and real images. Experiments with YOLOv8, YOLOv9 and RT-DETR reveal that training with synthetic data can achieve improved detection accuracies (at minimal costs). A human expert survey demonstrates the high realism of generated images, with experts not capable to distinguish them from real microscopy images (accuracy 50%). Our findings suggest that diffusion-based synthetic data generation is a promising avenue for augmenting real datasets in microscopy image analysis, reducing the reliance on extensive manual annotation and potentially improving the robustness of cell detection models.