FEDMEKI: A Benchmark for Scaling Medical Foundation Models via Federated Knowledge Injection

TL;DR

This paper introduces FEDMEKI, a privacy-preserving benchmark and platform for federated knowledge injection into medical foundation models, addressing the challenge of integrating multi-modal medical data without central data collection. It provides a curated, multi-site dataset spanning eight tasks and seven modalities, plus a 16-strong baseline suite to benchmark both training-task knowledge injection and zero-shot validation. The FedMeKI platform demonstrates that knowledge from private client data can be injected into a server-hosted medical FM using cross-silo FL, improving cross-task adaptability while highlighting trade-offs in performance versus traditional FL baselines. The work outlines practical implications for scalable, privacy-conscious medical AI and offers a foundation for future enhancements across more modalities and tasks while preserving patient privacy.

Abstract

This study introduces the Federated Medical Knowledge Injection (FEDMEKI) platform, a new benchmark designed to address the unique challenges of integrating medical knowledge into foundation models under privacy constraints. By leveraging a cross-silo federated learning approach, FEDMEKI circumvents the issues associated with centralized data collection, which is often prohibited under health regulations like the Health Insurance Portability and Accountability Act (HIPAA) in the USA. The platform is meticulously designed to handle multi-site, multi-modal, and multi-task medical data, which includes 7 medical modalities, including images, signals, texts, laboratory test results, vital signs, input variables, and output variables. The curated dataset to validate FEDMEKI covers 8 medical tasks, including 6 classification tasks (lung opacity detection, COVID-19 detection, electrocardiogram (ECG) abnormal detection, mortality prediction, sepsis prediction, and enlarged cardiomediastinum detection) and 2 generation tasks (medical visual question answering (MedVQA) and ECG noise clarification). This comprehensive dataset is partitioned across several clients to facilitate the decentralized training process under 16 benchmark approaches. FEDMEKI not only preserves data privacy but also enhances the capability of medical foundation models by allowing them to learn from a broader spectrum of medical knowledge without direct data exposure, thereby setting a new benchmark in the application of foundation models within the healthcare sector.

Figures (15)

• Figure 1: Overview of our proposed FedMeKI platform.
• Figure 2: Data sample of lung opacity detection.
• Figure 3: Data sample of Covid-19 detection.
• Figure 4: Data sample of ECG abnormal detection.
• Figure 5: Data sample of mortality prediction.