Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

RadSAM: Segmenting 3D radiological images with a 2D promptable model

Julien Khlaut, Elodie Ferreres, Daniel Tordjman, Hélène Philippe, Tom Boeken, Pierre Manceron, Corentin Dancette

TL;DR

RadSAM addresses the gap in 3D radiological segmentation by enabling 3D object recovery from a single 2D prompt using a novel mask-prompt and an iterative inference pipeline. Built on the Segment Anything Model architecture, RadSAM trains with multiple prompt types and a dedicated edition mechanism to refine 3D masks slice-by-slice while maintaining a 2D memory footprint. The approach demonstrates strong performance on AMOS and transferability to TotalSegmentator, with clear gains from editing and larger pretraining, and it shows promise for plaque-free, interactive clinical workflows. This work advances practical 3D medical image segmentation by combining promptable 2D models with volume-aware inference and user-driven corrections, enabling faster, more accurate radiology analysis.

Abstract

Medical image segmentation is a crucial and time-consuming task in clinical care, where mask precision is extremely important. The Segment Anything Model (SAM) offers a promising approach, as it provides an interactive interface based on visual prompting and edition to refine an initial segmentation. This model has strong generalization capabilities, does not rely on predefined classes, and adapts to diverse objects; however, it is pre-trained on natural images and lacks the ability to process medical data effectively. In addition, this model is built for 2D images, whereas a whole medical domain is based on 3D images, such as CT and MRI. Recent adaptations of SAM for medical imaging are based on 2D models, thus requiring one prompt per slice to segment 3D objects, making the segmentation process tedious. They also lack important features such as editing. To bridge this gap, we propose RadSAM, a novel method for segmenting 3D objects with a 2D model from a single prompt. In practice, we train a 2D model using noisy masks as initial prompts, in addition to bounding boxes and points. We then use this novel prompt type with an iterative inference pipeline to reconstruct the 3D mask slice-by-slice. We introduce a benchmark to evaluate the model's ability to segment 3D objects in CT images from a single prompt and evaluate the models' out-of-domain transfer and edition capabilities. We demonstrate the effectiveness of our approach against state-of-the-art models on this benchmark using the AMOS abdominal organ segmentation dataset.

RadSAM: Segmenting 3D radiological images with a 2D promptable model

TL;DR

RadSAM addresses the gap in 3D radiological segmentation by enabling 3D object recovery from a single 2D prompt using a novel mask-prompt and an iterative inference pipeline. Built on the Segment Anything Model architecture, RadSAM trains with multiple prompt types and a dedicated edition mechanism to refine 3D masks slice-by-slice while maintaining a 2D memory footprint. The approach demonstrates strong performance on AMOS and transferability to TotalSegmentator, with clear gains from editing and larger pretraining, and it shows promise for plaque-free, interactive clinical workflows. This work advances practical 3D medical image segmentation by combining promptable 2D models with volume-aware inference and user-driven corrections, enabling faster, more accurate radiology analysis.

Abstract

Medical image segmentation is a crucial and time-consuming task in clinical care, where mask precision is extremely important. The Segment Anything Model (SAM) offers a promising approach, as it provides an interactive interface based on visual prompting and edition to refine an initial segmentation. This model has strong generalization capabilities, does not rely on predefined classes, and adapts to diverse objects; however, it is pre-trained on natural images and lacks the ability to process medical data effectively. In addition, this model is built for 2D images, whereas a whole medical domain is based on 3D images, such as CT and MRI. Recent adaptations of SAM for medical imaging are based on 2D models, thus requiring one prompt per slice to segment 3D objects, making the segmentation process tedious. They also lack important features such as editing. To bridge this gap, we propose RadSAM, a novel method for segmenting 3D objects with a 2D model from a single prompt. In practice, we train a 2D model using noisy masks as initial prompts, in addition to bounding boxes and points. We then use this novel prompt type with an iterative inference pipeline to reconstruct the 3D mask slice-by-slice. We introduce a benchmark to evaluate the model's ability to segment 3D objects in CT images from a single prompt and evaluate the models' out-of-domain transfer and edition capabilities. We demonstrate the effectiveness of our approach against state-of-the-art models on this benchmark using the AMOS abdominal organ segmentation dataset.
Paper Structure (34 sections, 3 equations, 11 figures, 11 tables)

This paper contains 34 sections, 3 equations, 11 figures, 11 tables.

Table of Contents

  1. Introduction
  2. Benchmark
  3. Datasets
  4. Input prompts
  5. RadSam Training
  6. Architecture
  7. Preprocessing and data curation
  8. Prompts generation
  9. Training procedure
  10. RadSAM with Volume-Level Prompting
  11. Iterative Segmentation
  12. Edition in iterative segmentation
  13. Results
  14. Main Results
  15. Volume-Level Prompting
  16. ...and 19 more sections

Figures (11)

  • Figure 1: Prompting strategies. Slice-level prompting consists of feeding the model with one prompt for each slice. Volume-level prompting consists of a single 2D prompt (point or bbox) and annotations for the top and bottom slices.
  • Figure 2: RadSAM Architecture. The available prompts are masks, boxes, and points.
  • Figure 3: RadSAM edition training pipeline. The mask decoder outputs a mask refined through editing points using the ground truth mask. The outputted mask is represented in yellow, the prompt in red, the ground truth mask in green, and the intersection between the ground truth and the outputted mask is in blue.
  • Figure 4: Examples of each prompt: mask, bounding box, and point.
  • Figure 5: The iterative segmentation pipeline. The model is applied to each slice independently, starting from the slice where the first prompt is. Then, the mask output is passed as a prompt to segment the next slice.
  • ...and 6 more figures