When SAM Meets Medical Images: An Investigation of Segment Anything Model (SAM) on Multi-phase Liver Tumor Segmentation

TL;DR

The paper investigates the Segment Anything Model (SAM) for multi-phase liver tumor segmentation (MPLiTS), evaluating zero-shot segmentation across prompt counts, image resolutions, and phase aggregation. It finds a noticeable gap between SAM and task-specific performance, especially with few prompts, but также demonstrates SAM's strength as an interactive annotation tool when guided by user prompts. The study uses a sizable in-house MPLiTS dataset and a thorough experimental setup to reveal how prompts, resolution, and phase fusion affect performance. Overall, SAM shows potential to accelerate medical image annotation and provides guidance for future work to close the segmentation performance gap in MPLiTS.

Abstract

Learning to segmentation without large-scale samples is an inherent capability of human. Recently, Segment Anything Model (SAM) performs the significant zero-shot image segmentation, attracting considerable attention from the computer vision community. Here, we investigate the capability of SAM for medical image analysis, especially for multi-phase liver tumor segmentation (MPLiTS), in terms of prompts, data resolution, phases. Experimental results demonstrate that there might be a large gap between SAM and expected performance. Fortunately, the qualitative results show that SAM is a powerful annotation tool for the community of interactive medical image segmentation.

Figures (4)

• Figure 1: Visual example of results with the various $\mathcal{P}$.
• Figure 2: Visual example of results with the various $\mathcal{R}$.
• Figure 3: Visual example of results with the various $\mathcal{R}$.
• Figure 4: Visual example of the entire result with $\mathcal{T}^{\mathcal{M}}_{10, 512}$.