Fine-Tuning Foundation Models with Federated Learning for Privacy Preserving Medical Time Series Forecasting

TL;DR

This work addresses privacy-preserving medical time-series forecasting by fine-tuning foundation models (Chronos) with Federated Learning across ECG and ICG data. It systematically compares FedAvg, FedProx, and Fed-LA, plus a local adaptation baseline, under IID and non-IID partitions using Vital Signs and PTB-XL datasets with a 64-step forecast horizon and $R=30$ rounds. Key finding: FL can outperform zero-shot Chronos and local training in certain heterogeneous settings, but local fine-tuning often yields the best results when client data are IID, illustrating nuanced trade-offs. The study highlights the need for robust FL techniques and personalized/federated meta-learning approaches to reliably adapt foundation models in privacy-sensitive medical contexts.

Abstract

Federated Learning (FL) provides a decentralized machine learning approach, where multiple devices or servers collaboratively train a model without sharing their raw data, thus enabling data privacy. This approach has gained significant interest in academia and industry due to its privacy-preserving properties, which are particularly valuable in the medical domain where data availability is often protected under strict regulations. A relatively unexplored area is the use of FL to fine-tune Foundation Models (FMs) for time series forecasting, potentially enhancing model efficacy by overcoming data limitation while maintaining privacy. In this paper, we fine-tuned time series FMs with Electrocardiogram (ECG) and Impedance Cardiography (ICG) data using different FL techniques. We then examined various scenarios and discussed the challenges FL faces under different data heterogeneity configurations. Our empirical results demonstrated that while FL can be effective for fine-tuning FMs on time series forecasting tasks, its benefits depend on the data distribution across clients. We highlighted the trade-offs in applying FL to FM fine-tuning.

Figures (4)

• Figure 1: Overview of fine-tuning a Foundation Model using Federated Learning at round $r$. The global server sends model weights $\theta_r$ to clients, who update them locally on their data. Clients return updated models $\theta_{i,r+1}$, which the server aggregates using weighted averaging to obtain $\theta_{r+1}$.
• Figure 2: An overview of a hybrid training architecture that combines Federated Learning and Local Adaptation.
• Figure 3: Comparisons of FedAvg, FedProx, Fed-LA, and local fine-tuning models in terms of RMSE and MAE against the number of rounds when the data distribution amongst clients is IID. All FL algorithms were outperformed by the locally fine-tuned models.
• Figure 4: Comparisons of FedAvg, FedProx, Fed-LA, and local fine-tuning models in terms of RMSE and MAE against the number of rounds when the data distribution amongst clients is non-IID. FL algorithms outperformed zero-shot and local fine-tuning.