Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Zero-shot Domain Generalization of Foundational Models for 3D Medical Image Segmentation: An Experimental Study

Soumitri Chattopadhyay, Basar Demir, Marc Niethammer

TL;DR

This study examines foundation models' ability towards domain generalization (DG), by conducting a comprehensive experimental study encompassing 6 medical segmentation FMs and 12 public datasets spanning multiple modalities and anatomies.

Abstract

Domain shift, caused by variations in imaging modalities and acquisition protocols, limits model generalization in medical image segmentation. While foundation models (FMs) trained on diverse large-scale data hold promise for zero-shot generalization, their application to volumetric medical data remains underexplored. In this study, we examine their ability towards domain generalization (DG), by conducting a comprehensive experimental study encompassing 6 medical segmentation FMs and 12 public datasets spanning multiple modalities and anatomies. Our findings reveal the potential of promptable FMs in bridging the domain gap via smart prompting techniques. Additionally, by probing into multiple facets of zero-shot DG, we offer valuable insights into the viability of FMs for DG and identify promising avenues for future research.

Zero-shot Domain Generalization of Foundational Models for 3D Medical Image Segmentation: An Experimental Study

TL;DR

This study examines foundation models' ability towards domain generalization (DG), by conducting a comprehensive experimental study encompassing 6 medical segmentation FMs and 12 public datasets spanning multiple modalities and anatomies.

Abstract

Domain shift, caused by variations in imaging modalities and acquisition protocols, limits model generalization in medical image segmentation. While foundation models (FMs) trained on diverse large-scale data hold promise for zero-shot generalization, their application to volumetric medical data remains underexplored. In this study, we examine their ability towards domain generalization (DG), by conducting a comprehensive experimental study encompassing 6 medical segmentation FMs and 12 public datasets spanning multiple modalities and anatomies. Our findings reveal the potential of promptable FMs in bridging the domain gap via smart prompting techniques. Additionally, by probing into multiple facets of zero-shot DG, we offer valuable insights into the viability of FMs for DG and identify promising avenues for future research.

Paper Structure

This paper contains 12 sections, 4 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Materials and Methods
  4. Datasets
  5. Model Taxonomy
  6. Experiments
  7. Zero-shot transfer of domain-specific FMs
  8. Visual promptable FMs
  9. Text promptable FMs
  10. Conclusion
  11. Acknowledgements
  12. Organ-wise segmentation results

Figures (4)

  • Figure 1: Qualitative results showing segmentation improvement in SegVol with text + bbox prompts for seen (CT; MSD-Spleen) and unseen (MR; CHAOS) modalities.
  • Figure 2: Box plots for organ-wise dice scores for all comparative models on the abdominal CT datasets used in this study.
  • Figure 3: Box plots for organ-wise dice scores for all comparative models on the abdominal MR datasets used in this study.
  • Figure 4: Box plots for organ-wise dice scores for all comparative models on the cardiac datasets used in this study.