Segment Anything for Dendrites from Electron Microscopy

TL;DR

This study introduces the first implementation of vision foundation models in dendrite segmentation, paving the path for computer-assisted diagnosis of neuronal anomalies in EM images.

Abstract

Segmentation of cellular structures in electron microscopy (EM) images is fundamental to analyzing the morphology of neurons and glial cells in the healthy and diseased brain tissue. Current neuronal segmentation applications are based on convolutional neural networks (CNNs) and do not effectively capture global relationships within images. Here, we present DendriteSAM, a vision foundation model based on Segment Anything, for interactive and automatic segmentation of dendrites in EM images. The model is trained on high-resolution EM data from healthy rat hippocampus and is tested on diseased rat and human data. Our evaluation results demonstrate better mask quality compared to the original and other fine-tuned models, leveraging the features learned during training. This study introduces the first implementation of vision foundation models in dendrite segmentation, paving the path for computer-assisted diagnosis of neuronal anomalies.

Figures (9)

• Figure 1: Concavity Distribution: The x-axis and y-axis represent the value of concavity and the corresponding percentage of masks in sampled images, respectively. The green star and red arrow in the subplot indicate the dendrite and spine structure, respectively.
• Figure 2: Model Architecture
• Figure 3: Impact of Image Tiling vs Resizing: On the left, segmentation quality with the model trained with the tiling method (cropping with a sliding window size 1024 and step size 512), and on the right with the model trained with the resizing method (resizing the whole image to 1024 $\times$ 1024). In the legend, the digits after "p" and "n" stand for the number of foreground points and background points used, respectively.
• Figure 4: Quantitative Evaluation: The blue and black legends present ViT-B and ViT-L models from SAM. ViT-B-EM-organelles weight1 and ViT-L-EM-organellesweight2 stand for the models from Micro_SAM trained on EM organelles images, and ViT-B-resize-EM-dendrite and ViT-L-resize-EM-dendrite are specialist models that we trained.
• Figure 5: Qualitative Analysis: Expert graded quality of randomly sampled masks predicted by ViT-L-resize-EM-dendrite with bbox_p4_n8 prompts.