Can SAM Segment Polyps?

TL;DR

This paper evaluates the Segment Anything Model (SAM) for polyp segmentation in colonoscopy images under unprompted prompts. It benchmarks SAM-H and SAM-L against 14 CNN-based and 2 Transformer-based baselines across five public datasets using six segmentation metrics, revealing that SAM lags behind specialized models. The study notes both quantitative and qualitative failures, particularly with blurred boundaries and domain mismatch between natural and medical images. It suggests fine-tuning SAM on polyp data as a promising path and advocates for further research into task-specific adaptation of general segmentation models for medical imaging.

Abstract

Recently, Meta AI Research releases a general Segment Anything Model (SAM), which has demonstrated promising performance in several segmentation tasks. As we know, polyp segmentation is a fundamental task in the medical imaging field, which plays a critical role in the diagnosis and cure of colorectal cancer. In particular, applying SAM to the polyp segmentation task is interesting. In this report, we evaluate the performance of SAM in segmenting polyps, in which SAM is under unprompted settings. We hope this report will provide insights to advance this polyp segmentation field and promote more interesting works in the future. This project is publicly at https://github.com/taozh2017/SAMPolyp.

Figures (2)

• Figure 1: Some better segmentation examples of SAM.
• Figure 2: Some failure segmentation examples of SAM.