Segment Anything Model (SAM) for Digital Pathology: Assess Zero-shot Segmentation on Whole Slide Imaging

TL;DR

This work assesses the zero-shot segmentation capabilities of the Segment Anything Model (SAM) on digital pathology whole slide imaging tasks (tumor, tissue, and cell nuclei segmentation). Using TCGA, NEPTUNE, and MoNuSeg datasets, SAM is evaluated under varying prompts and compared with state-of-the-art methods, revealing strong performance on large, connected structures but gaps in dense object segmentation. The study identifies critical limitations—image resolution, multi-scale needs, prompt selection, and lack of fine-tuning—highlighting that few-shot adaptation may be essential to fully realize SAM's potential in WSI analysis. Overall, SAM offers promising zero-shot utility for digital pathology, but practical deployment will require targeted fine-tuning and refined prompting strategies to handle complex, dense tissue scenes.

Abstract

The segment anything model (SAM) was released as a foundation model for image segmentation. The promptable segmentation model was trained by over 1 billion masks on 11M licensed and privacy-respecting images. The model supports zero-shot image segmentation with various segmentation prompts (e.g., points, boxes, masks). It makes the SAM attractive for medical image analysis, especially for digital pathology where the training data are rare. In this study, we evaluate the zero-shot segmentation performance of SAM model on representative segmentation tasks on whole slide imaging (WSI), including (1) tumor segmentation, (2) non-tumor tissue segmentation, (3) cell nuclei segmentation. Core Results: The results suggest that the zero-shot SAM model achieves remarkable segmentation performance for large connected objects. However, it does not consistently achieve satisfying performance for dense instance object segmentation, even with 20 prompts (clicks/boxes) on each image. We also summarized the identified limitations for digital pathology: (1) image resolution, (2) multiple scales, (3) prompt selection, and (4) model fine-tuning. In the future, the few-shot fine-tuning with images from downstream pathological segmentation tasks might help the model to achieve better performance in dense object segmentation.

Figures (1)

• Figure 1: Qualitative segmentation results. The SOTA methods are compared with SAM method with different prompt strategies.