Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Leveraging SAM for Single-Source Domain Generalization in Medical Image Segmentation

Hanhui Wang, Huaize Ye, Yi Xia, Xueyan Zhang

TL;DR

The paper addresses the challenge of domain shifts in medical image segmentation under single-source domain generalization. It proposes a parallel framework that combines a traditional segmentation backbone with a fine-tuned Segment Anything Model (SAM), augmented by a mask-filtering postprocess and an image-merging strategy to generate high-quality prompts efficiently. By deriving refined bounding boxes from coarse CNN masks and training only SAM's mask decoder, the approach achieves state-of-the-art Dice scores on the Prostate MRI dataset across six unseen domains, notably reaching 79.54% in cross-domain evaluation. Overall, the method offers a practical, efficient pathway to robust SDG in medical imaging with potential applicability to broader segmentation tasks.

Abstract

Domain Generalization (DG) aims to reduce domain shifts between domains to achieve promising performance on the unseen target domain, which has been widely practiced in medical image segmentation. Single-source domain generalization (SDG) is the most challenging setting that trains on only one source domain. Although existing methods have made considerable progress on SDG of medical image segmentation, the performances are still far from the applicable standards when faced with a relatively large domain shift. In this paper, we leverage the Segment Anything Model (SAM) to SDG to greatly improve the ability of generalization. Specifically, we introduce a parallel framework, the source images are sent into the SAM module and normal segmentation module respectively. To reduce the calculation resources, we apply a merging strategy before sending images to the SAM module. We extract the bounding boxes from the segmentation module and send the refined version as prompts to the SAM module. We evaluate our model on a classic DG dataset and achieve competitive results compared to other state-of-the-art DG methods. Furthermore, We conducted a series of ablation experiments to prove the effectiveness of the proposed method. The code is publicly available at https://github.com/SARIHUST/SAMMed.

Leveraging SAM for Single-Source Domain Generalization in Medical Image Segmentation

TL;DR

The paper addresses the challenge of domain shifts in medical image segmentation under single-source domain generalization. It proposes a parallel framework that combines a traditional segmentation backbone with a fine-tuned Segment Anything Model (SAM), augmented by a mask-filtering postprocess and an image-merging strategy to generate high-quality prompts efficiently. By deriving refined bounding boxes from coarse CNN masks and training only SAM's mask decoder, the approach achieves state-of-the-art Dice scores on the Prostate MRI dataset across six unseen domains, notably reaching 79.54% in cross-domain evaluation. Overall, the method offers a practical, efficient pathway to robust SDG in medical imaging with potential applicability to broader segmentation tasks.

Abstract

Domain Generalization (DG) aims to reduce domain shifts between domains to achieve promising performance on the unseen target domain, which has been widely practiced in medical image segmentation. Single-source domain generalization (SDG) is the most challenging setting that trains on only one source domain. Although existing methods have made considerable progress on SDG of medical image segmentation, the performances are still far from the applicable standards when faced with a relatively large domain shift. In this paper, we leverage the Segment Anything Model (SAM) to SDG to greatly improve the ability of generalization. Specifically, we introduce a parallel framework, the source images are sent into the SAM module and normal segmentation module respectively. To reduce the calculation resources, we apply a merging strategy before sending images to the SAM module. We extract the bounding boxes from the segmentation module and send the refined version as prompts to the SAM module. We evaluate our model on a classic DG dataset and achieve competitive results compared to other state-of-the-art DG methods. Furthermore, We conducted a series of ablation experiments to prove the effectiveness of the proposed method. The code is publicly available at https://github.com/SARIHUST/SAMMed.
Paper Structure (15 sections, 2 figures, 3 tables)

This paper contains 15 sections, 2 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Segment Anything Model
  4. Domain Generalization & Single-Source Domain Generalization
  5. Methods
  6. Architecture Overview
  7. Segmentation Backbone
  8. Mask-Filtering Module
  9. SAM Section
  10. Experiments
  11. Experiment Setting
  12. Evaluation on Prostate
  13. Ablation Studies
  14. Qualitative Results
  15. Conclusion

Figures (2)

  • Figure 1: Overall pipeline of our proposed paradigm. Firstly, we utilize a traditional segmentation model to predict coarse masks, and then apply a mask-filtering module to produce refined bounding boxes, which are then used to fine-tune SAM.
  • Figure 2: Qualitative results on Domain E and F of the Prostate dataset. The left column demonstrates 4 instances from domain E, and the right column demonstrates 4 instances from domain F. The first row demonstrates the source image for segmentation, the second row demonstrates the coarse masks predicted by the Resnet we use as the segmentation model, the third row demonstrates the final mask results predicted by our fine-tuned SAM, and the last row demonstrates the ground truth masks.