Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

FedFMS: Exploring Federated Foundation Models for Medical Image Segmentation

Yuxi Liu, Guibo Luo, Yuesheng Zhu

TL;DR

Federated Foundation models for Medical image Segmentation (FedFMS), which includes the Federated SAM (FedSAM) and a communication and training-efficient Federated SAM with Medical SAM Adapter (FedMSA), revealed that FedFMS could achieve performance comparable to models trained via centralized training methods while maintaining privacy.

Abstract

Medical image segmentation is crucial for clinical diagnosis. The Segmentation Anything Model (SAM) serves as a powerful foundation model for visual segmentation and can be adapted for medical image segmentation. However, medical imaging data typically contain privacy-sensitive information, making it challenging to train foundation models with centralized storage and sharing. To date, there are few foundation models tailored for medical image deployment within the federated learning framework, and the segmentation performance, as well as the efficiency of communication and training, remain unexplored. In response to these issues, we developed Federated Foundation models for Medical image Segmentation (FedFMS), which includes the Federated SAM (FedSAM) and a communication and training-efficient Federated SAM with Medical SAM Adapter (FedMSA). Comprehensive experiments on diverse datasets are conducted to investigate the performance disparities between centralized training and federated learning across various configurations of FedFMS. The experiments revealed that FedFMS could achieve performance comparable to models trained via centralized training methods while maintaining privacy. Furthermore, FedMSA demonstrated the potential to enhance communication and training efficiency. Our model implementation codes are available at https://github.com/LIU-YUXI/FedFMS.

FedFMS: Exploring Federated Foundation Models for Medical Image Segmentation

TL;DR

Federated Foundation models for Medical image Segmentation (FedFMS), which includes the Federated SAM (FedSAM) and a communication and training-efficient Federated SAM with Medical SAM Adapter (FedMSA), revealed that FedFMS could achieve performance comparable to models trained via centralized training methods while maintaining privacy.

Abstract

Medical image segmentation is crucial for clinical diagnosis. The Segmentation Anything Model (SAM) serves as a powerful foundation model for visual segmentation and can be adapted for medical image segmentation. However, medical imaging data typically contain privacy-sensitive information, making it challenging to train foundation models with centralized storage and sharing. To date, there are few foundation models tailored for medical image deployment within the federated learning framework, and the segmentation performance, as well as the efficiency of communication and training, remain unexplored. In response to these issues, we developed Federated Foundation models for Medical image Segmentation (FedFMS), which includes the Federated SAM (FedSAM) and a communication and training-efficient Federated SAM with Medical SAM Adapter (FedMSA). Comprehensive experiments on diverse datasets are conducted to investigate the performance disparities between centralized training and federated learning across various configurations of FedFMS. The experiments revealed that FedFMS could achieve performance comparable to models trained via centralized training methods while maintaining privacy. Furthermore, FedMSA demonstrated the potential to enhance communication and training efficiency. Our model implementation codes are available at https://github.com/LIU-YUXI/FedFMS.
Paper Structure (18 sections, 2 figures, 2 tables)

This paper contains 18 sections, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Method
  3. Preliminary Methods
  4. SAM Architecture
  5. MSA Architecture
  6. Federated Foundation Models
  7. Client-side Model Training
  8. Server-side Model Aggregation
  9. Dataset Preparation
  10. Experiments
  11. Experimental Settings
  12. Results
  13. Overall Comparison
  14. Model Efficiency Analysis
  15. Pre-training Impact
  16. ...and 3 more sections

Figures (2)

  • Figure 1: The illustration of FedSAM and FedMSA. FedSAM is a federated learning framework with (a) SAM. FedMSA is a federated learning framework with (b) MSA.
  • Figure 2: The example of images and Ground Truth from different clients across four datasets (a-d) and the comparison in the results of FedMSA, MSA, FedSAM and SAM. The bottom right corner of each image indicates the Dice for it or the whole nii.