Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

FACMIC: Federated Adaptative CLIP Model for Medical Image Classification

Yihang Wu, Christian Desrosiers, Ahmad Chaddad

TL;DR

A federated adaptive Contrastive Language Image Pretraining CLIP model designed for classification tasks is introduced that employs a light-weight and efficient feature attention module for CLIP that selects suitable features for each client's data.

Abstract

Federated learning (FL) has emerged as a promising approach to medical image analysis that allows deep model training using decentralized data while ensuring data privacy. However, in the field of FL, communication cost plays a critical role in evaluating the performance of the model. Thus, transferring vision foundation models can be particularly challenging due to the significant resource costs involved. In this paper, we introduce a federated adaptive Contrastive Language Image Pretraining CLIP model designed for classification tasks. We employ a light-weight and efficient feature attention module for CLIP that selects suitable features for each client's data. Additionally, we propose a domain adaptation technique to reduce differences in data distribution between clients. Experimental results on four publicly available datasets demonstrate the superior performance of FACMIC in dealing with real-world and multisource medical imaging data. Our codes are available at https://github.com/AIPMLab/FACMIC.

FACMIC: Federated Adaptative CLIP Model for Medical Image Classification

TL;DR

A federated adaptive Contrastive Language Image Pretraining CLIP model designed for classification tasks is introduced that employs a light-weight and efficient feature attention module for CLIP that selects suitable features for each client's data.

Abstract

Federated learning (FL) has emerged as a promising approach to medical image analysis that allows deep model training using decentralized data while ensuring data privacy. However, in the field of FL, communication cost plays a critical role in evaluating the performance of the model. Thus, transferring vision foundation models can be particularly challenging due to the significant resource costs involved. In this paper, we introduce a federated adaptive Contrastive Language Image Pretraining CLIP model designed for classification tasks. We employ a light-weight and efficient feature attention module for CLIP that selects suitable features for each client's data. Additionally, we propose a domain adaptation technique to reduce differences in data distribution between clients. Experimental results on four publicly available datasets demonstrate the superior performance of FACMIC in dealing with real-world and multisource medical imaging data. Our codes are available at https://github.com/AIPMLab/FACMIC.

Paper Structure

This paper contains 12 sections, 8 equations, 2 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Material and methods
  3. Problem formulation
  4. Our federated adaptive CLIP framework
  5. Experiments
  6. Datasets
  7. Implementation details
  8. Comparison with state-of-the-art methods
  9. Ablation study
  10. Conclusion
  11. Acknowledgments.
  12. Disclosure of Interests.

Figures (2)

  • Figure 1: The proposed FACMIC framework. Each client trains its model separately, optimizing only the parameters of its local attention module ($a_i$) using contrastive and domain adaptation losses. After receiving the local client parameters, the server aggregates them into a global attention module ($a_{\mathit{global}}$) whose parameters are broadcasted back to clients.
  • Figure 2: Global testing accuracy (%) for each round on the BT dataset.