Federated Learning for Time-Series Healthcare Sensing with Incomplete Modalities

TL;DR

FLISM is an efficient FL training algorithm with incomplete sensing modalities while maintaining high accuracy, and is faster and more efficient compared with state-of-the-art methods handling incomplete modality problems in FL.

Abstract

Many healthcare sensing applications utilize multimodal time-series data from sensors embedded in mobile and wearable devices. Federated Learning (FL), with its privacy-preserving advantages, is particularly well-suited for health applications. However, most multimodal FL methods assume the availability of complete modality data for local training, which is often unrealistic. Moreover, recent approaches tackling incomplete modalities scale poorly and become inefficient as the number of modalities increases. To address these limitations, we propose FLISM, an efficient FL training algorithm with incomplete sensing modalities while maintaining high accuracy. FLISM employs three key techniques: (1) modality-invariant representation learning to extract effective features from clients with a diverse set of modalities, (2) modality quality-aware aggregation to prioritize contributions from clients with higher-quality modality data, and (3) global-aligned knowledge distillation to reduce local update shifts caused by modality differences. Extensive experiments on real-world datasets show that FLISM not only achieves high accuracy but is also faster and more efficient compared with state-of-the-art methods handling incomplete modality problems in FL. We release the code as open-source at https://github.com/AdibaOrz/FLISM.

Figures (8)

• Figure 1: Overview of FLISM, consisting of three key components: ①: Modality-Invariant Representation Learning (MIRL) learns to extract effective features, ②: Modality Quality-Aware Aggregation (MQAA) priorities clients with higher-quality modality data, and ③: Global-Aligned Knowledge Distillation (GAKD) reduces deviations in client updates by aligning the predictions of the local model with that of the global.
• Figure 2: Comparison of FLISM with other baselines on communication and computation cost.
• Figure 3: Scalability analysis of FLISM: Comparing communication (left) and computation (right) costs.
• Figure 4: Comparison of resource usage between intermediate and early fusion based on the number of MACs (left) and model parameters (right).
• Figure 5: Deep imputation's extra training burden represented by the number of cross-modality transfer models (left) and the corresponding MACs (right).