Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SAM-Fed: SAM-Guided Federated Semi-Supervised Learning for Medical Image Segmentation

Sahar Nasirihaghighi, Negin Ghamsarian, Yiping Li, Marcel Breeuwer, Raphael Sznitman, Klaus Schoeffmann

TL;DR

This paper tackles the challenge of data privacy and limited labeled data in medical image segmentation by introducing SAM-Fed, a federated semi-supervised framework that leverages a high-capacity server-side segmentation model (SAM) to guide lightweight clients. It combines dual knowledge distillation—federated knowledge transfer between global and local models and SAM-guided pixel-level supervision for small models—with an adaptive pixel-wise agreement mechanism to produce reliable pseudo-labels from unlabeled client data. The method supports both homogeneous (FedAvg) and heterogeneous (RC+RF) aggregations and demonstrates strong improvements over state-of-the-art FSSL baselines on ISIC2018 skin lesions and polyp segmentation under varied client architectures and non-IID settings. The results indicate that server-guided supervision can bridge the gap between model capacity and client constraints, enabling robust, privacy-preserving medical image segmentation in real-world clinical deployments.

Abstract

Medical image segmentation is clinically important, yet data privacy and the cost of expert annotation limit the availability of labeled data. Federated semi-supervised learning (FSSL) offers a solution but faces two challenges: pseudo-label reliability depends on the strength of local models, and client devices often require compact or heterogeneous architectures due to limited computational resources. These constraints reduce the quality and stability of pseudo-labels, while large models, though more accurate, cannot be trained or used for routine inference on client devices. We propose SAM-Fed, a federated semi-supervised framework that leverages a high-capacity segmentation foundation model to guide lightweight clients during training. SAM-Fed combines dual knowledge distillation with an adaptive agreement mechanism to refine pixel-level supervision. Experiments on skin lesion and polyp segmentation across homogeneous and heterogeneous settings show that SAM-Fed consistently outperforms state-of-the-art FSSL methods.

SAM-Fed: SAM-Guided Federated Semi-Supervised Learning for Medical Image Segmentation

TL;DR

This paper tackles the challenge of data privacy and limited labeled data in medical image segmentation by introducing SAM-Fed, a federated semi-supervised framework that leverages a high-capacity server-side segmentation model (SAM) to guide lightweight clients. It combines dual knowledge distillation—federated knowledge transfer between global and local models and SAM-guided pixel-level supervision for small models—with an adaptive pixel-wise agreement mechanism to produce reliable pseudo-labels from unlabeled client data. The method supports both homogeneous (FedAvg) and heterogeneous (RC+RF) aggregations and demonstrates strong improvements over state-of-the-art FSSL baselines on ISIC2018 skin lesions and polyp segmentation under varied client architectures and non-IID settings. The results indicate that server-guided supervision can bridge the gap between model capacity and client constraints, enabling robust, privacy-preserving medical image segmentation in real-world clinical deployments.

Abstract

Medical image segmentation is clinically important, yet data privacy and the cost of expert annotation limit the availability of labeled data. Federated semi-supervised learning (FSSL) offers a solution but faces two challenges: pseudo-label reliability depends on the strength of local models, and client devices often require compact or heterogeneous architectures due to limited computational resources. These constraints reduce the quality and stability of pseudo-labels, while large models, though more accurate, cannot be trained or used for routine inference on client devices. We propose SAM-Fed, a federated semi-supervised framework that leverages a high-capacity segmentation foundation model to guide lightweight clients during training. SAM-Fed combines dual knowledge distillation with an adaptive agreement mechanism to refine pixel-level supervision. Experiments on skin lesion and polyp segmentation across homogeneous and heterogeneous settings show that SAM-Fed consistently outperforms state-of-the-art FSSL methods.

Paper Structure

This paper contains 5 sections, 4 equations, 2 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Proposed Approach
  3. Experimental Settings
  4. Experimental Results
  5. Conclusion

Figures (2)

  • Figure 1: Overview of SAM-Fed.
  • Figure 2: Adaptive agreement mechanism for pseudo-label generation, with examples from early (row 1), middle (row 2), and final (row 3) training epochs. The agreement mask highlights pixel-wise consistency between predictions (green = agreement, red = disagreement).