Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SAM & SAM 2 in 3D Slicer: SegmentWithSAM Extension for Annotating Medical Images

Zafer Yildiz, Yuwen Chen, Maciej A. Mazurowski

TL;DR

Medical image annotation is slow and requires experts. The authors adapt SAM and SAM 2 to 3D Slicer, enabling interactive 2D prompt-based segmentation and 3D propagation across volumes. The work supports multiple checkpoints, precomputes per-slice embeddings, and offers two propagation modes plus refinement with 3D Slicer tools, all available as an open-source extension. The resulting workflow reduces annotation time and supports multi-modality data, with easy installation from the Extension Manager.

Abstract

Creating annotations for 3D medical data is time-consuming and often requires highly specialized expertise. Various tools have been implemented to aid this process. Segment Anything Model 2 (SAM 2) offers a general-purpose prompt-based segmentation algorithm designed to annotate videos. In this paper, we adapt this model to the annotation of 3D medical images and offer our implementation in the form of an extension to the popular annotation software: 3D Slicer. Our extension allows users to place point prompts on 2D slices to generate annotation masks and propagate these annotations across entire volumes in either single-directional or bi-directional manners. Our code is publicly available on https://github.com/mazurowski-lab/SlicerSegmentWithSAM and can be easily installed directly from the Extension Manager of 3D Slicer as well.

SAM & SAM 2 in 3D Slicer: SegmentWithSAM Extension for Annotating Medical Images

TL;DR

Medical image annotation is slow and requires experts. The authors adapt SAM and SAM 2 to 3D Slicer, enabling interactive 2D prompt-based segmentation and 3D propagation across volumes. The work supports multiple checkpoints, precomputes per-slice embeddings, and offers two propagation modes plus refinement with 3D Slicer tools, all available as an open-source extension. The resulting workflow reduces annotation time and supports multi-modality data, with easy installation from the Extension Manager.

Abstract

Creating annotations for 3D medical data is time-consuming and often requires highly specialized expertise. Various tools have been implemented to aid this process. Segment Anything Model 2 (SAM 2) offers a general-purpose prompt-based segmentation algorithm designed to annotate videos. In this paper, we adapt this model to the annotation of 3D medical images and offer our implementation in the form of an extension to the popular annotation software: 3D Slicer. Our extension allows users to place point prompts on 2D slices to generate annotation masks and propagate these annotations across entire volumes in either single-directional or bi-directional manners. Our code is publicly available on https://github.com/mazurowski-lab/SlicerSegmentWithSAM and can be easily installed directly from the Extension Manager of 3D Slicer as well.
Paper Structure (9 sections, 2 figures)

This paper contains 9 sections, 2 figures.

Table of Contents

  1. Introduction
  2. Methodology
  3. 2D Segmentation
  4. 3D Segmentation
  5. Propagation through all slices:
  6. Propagation to left or right:
  7. Refinement
  8. Results
  9. Conclusion

Figures (2)

  • Figure 1: Overview of pipeline for SegmentWithSAM
  • Figure 2: Performance of SegmentWithSAM on different modality samples