TriSAM: Tri-Plane SAM for zero-shot cortical blood vessel segmentation in VEM images

TL;DR

This work addresses the lack of a standardized benchmark for cortical blood vessel segmentation in Volume Electron Microscopy by introducing the BvEM dataset, spanning mouse, macaque, and human volumes with standardized resolution and expert-verified 3D vessel annotations. It then presents TriSAM, a zero-shot 3D segmentation framework that adapts the 2D Segment Anything Model through tri-plane plane selection, SAM-based tracking, and recursive turning-point sampling to grow vascular structures without training. Across the BvEM volumes, TriSAM outperforms zero-shot and supervised baselines, validating the approach's ability to handle diverse imaging appearances and complex 3D vessel morphology. By providing the dataset, code, and model online, the authors offer practical tools to advance microscale neurovascular analysis and contribute to understanding brain vascular networks.

Abstract

While imaging techniques at macro and mesoscales have garnered substantial attention and resources, microscale Volume Electron Microscopy (vEM) imaging, capable of revealing intricate vascular details, has lacked the necessary benchmarking infrastructure. In this paper, we address a significant gap in this field of neuroimaging by introducing the first-in-class public benchmark, BvEM, designed specifically for cortical blood vessel segmentation in vEM images. Our BvEM benchmark is based on vEM image volumes from three mammals: adult mouse, macaque, and human. We standardized the resolution, addressed imaging variations, and meticulously annotated blood vessels through semi-automatic, manual, and quality control processes, ensuring high-quality 3D segmentation. Furthermore, we developed a zero-shot cortical blood vessel segmentation method named TriSAM, which leverages the powerful segmentation model SAM for 3D segmentation. To extend SAM from 2D to 3D volume segmentation, TriSAM employs a multi-seed tracking framework, leveraging the reliability of certain image planes for tracking while using others to identify potential turning points. This approach effectively achieves long-term 3D blood vessel segmentation without model training or fine-tuning. Experimental results show that TriSAM achieved superior performances on the BvEM benchmark across three species. Our dataset, code, and model are available online at \url{https://jia-wan.github.io/bvem}.

Figures (12)

• Figure 1: Imaging modalities for blood vessel analysis. (a) Both microtomography ($\mu$CT) dyer2017quantifying and light microscopy (LM) lochhead2023high can only capture blood vessels in the cortex at the sub-micron resolution without ultrastructure details. (b) Volume electron microscopy (VEM) can show unbiased details of the vasculature including all types of cells at a higher resolution.
• Figure 2: The proposed BvEM dataset. We proofread the blood vessel instance segmentation (displayed in different colors) in the three largest publicly available VEM volumes: (a) mouse microns-phase2, macaque loomba2022connectomic, and human shapson2021connectomic samples acquired at different VEM labs.
• Figure 3: Dataset statistics. Empirical distribution of blood vessel radius.
• Figure 4: TriSAM framework. (a) Tri-plane selection is first proposed to select the best plane for tracking. (b) SAM-based tracking leverages SAM to perform short-term tracking given a seed location and a tracking axis. (c) Recursive seed sampling exploits potential turning points for long-term tracking.
• Figure 5: Qualitative instance segmentation results on the BvEM-Macaque volume. Different colors indicate different instances. Color thresholding and 3D UNet often produce false positives, whereas SAM+IoU tracking tends to miss a significant portion of blood vessels. Among the comparison methods, TriSAM segmentation stands out as the most effective.