Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Heterogeneous Federated Learning System for Sparse Healthcare Time-Series Prediction

Jia-Hao Syu, Jerry Chun-Wei Lin

TL;DR

This paper addresses predicting sparse healthcare time-series data across heterogeneous institutions while preserving privacy. It proposes Heterogeneous Federated Learning (HFL) that constructs dense feature tensors for feature-wise learning and sparse feature tensors for temporal embedding, with a global head per feature, a local embedding module, and a fusion predictor. Key contributions include heterogeneous domain selection, asynchronous switching to enable asynchronous transfers, and ablation showing the importance of model selection and switching; experimental results on MIMIC datasets show HFL achieves state-of-the-art MSE reductions on eight of ten tasks (e.g., MF5 MSE of 561.97 with reductions of $94.8 ext{%}$, $48.3 ext{%}$, and $52.1 ext{%}$ relative to DNN, BIBE, and BIBEP). The results demonstrate the effectiveness of heterogeneous knowledge transfer for small target domains and the potential for privacy-preserving deployment in real-world healthcare. Future work includes distributed computation and stronger security.

Abstract

In this paper, we propose a heterogeneous federated learning (HFL) system for sparse time series prediction in healthcare, which is a decentralized federated learning algorithm with heterogeneous transfers. We design dense and sparse feature tensors to deal with the sparsity of data sources. Heterogeneous federated learning is developed to share asynchronous parts of networks and select appropriate models for knowledge transfer. Experimental results show that the proposed HFL achieves the lowest prediction error among all benchmark systems on eight out of ten prediction tasks, with MSE reduction of 94.8%, 48.3%, and 52.1% compared to the benchmark systems. These results demonstrate the effectiveness of HFL in transferring knowledge from heterogeneous domains, especially in the smaller target domain. Ablation studies then demonstrate the effectiveness of the designed mechanisms for heterogeneous domain selection and switching in predicting healthcare time series with privacy, model security, and heterogeneous knowledge transfer.

Heterogeneous Federated Learning System for Sparse Healthcare Time-Series Prediction

TL;DR

This paper addresses predicting sparse healthcare time-series data across heterogeneous institutions while preserving privacy. It proposes Heterogeneous Federated Learning (HFL) that constructs dense feature tensors for feature-wise learning and sparse feature tensors for temporal embedding, with a global head per feature, a local embedding module, and a fusion predictor. Key contributions include heterogeneous domain selection, asynchronous switching to enable asynchronous transfers, and ablation showing the importance of model selection and switching; experimental results on MIMIC datasets show HFL achieves state-of-the-art MSE reductions on eight of ten tasks (e.g., MF5 MSE of 561.97 with reductions of , , and relative to DNN, BIBE, and BIBEP). The results demonstrate the effectiveness of heterogeneous knowledge transfer for small target domains and the potential for privacy-preserving deployment in real-world healthcare. Future work includes distributed computation and stronger security.

Abstract

In this paper, we propose a heterogeneous federated learning (HFL) system for sparse time series prediction in healthcare, which is a decentralized federated learning algorithm with heterogeneous transfers. We design dense and sparse feature tensors to deal with the sparsity of data sources. Heterogeneous federated learning is developed to share asynchronous parts of networks and select appropriate models for knowledge transfer. Experimental results show that the proposed HFL achieves the lowest prediction error among all benchmark systems on eight out of ten prediction tasks, with MSE reduction of 94.8%, 48.3%, and 52.1% compared to the benchmark systems. These results demonstrate the effectiveness of HFL in transferring knowledge from heterogeneous domains, especially in the smaller target domain. Ablation studies then demonstrate the effectiveness of the designed mechanisms for heterogeneous domain selection and switching in predicting healthcare time series with privacy, model security, and heterogeneous knowledge transfer.
Paper Structure (19 sections, 8 equations, 7 figures, 7 tables)

This paper contains 19 sections, 8 equations, 7 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Literature Review
  3. Healthcare Time-Series Prediction
  4. Transfer Learning
  5. Federated Learning
  6. Preliminary
  7. Sparse Feature Tensor
  8. Dense Feature Tensor
  9. Proposed Heterogeneous Federated Learning System (HFL)
  10. Network Design
  11. Heterogeneous Federated Learning
  12. Properties of HFL
  13. Experiments and Discussion
  14. Data Usage
  15. Experiment Setup
  16. ...and 4 more sections

Figures (7)

  • Figure 1: Heterogeneity and security issues in the healthcare industry
  • Figure 2: Sparse healthcare data
  • Figure 3: Packing of sparse feature tensors
  • Figure 4: Packing of dense feature tensors
  • Figure 5: Network design of the proposed HFL
  • ...and 2 more figures