Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

FlexFed: Mitigating Catastrophic Forgetting in Heterogeneous Federated Learning in Pervasive Computing Environments

Sara Alosaime, Arshad Jhumka

TL;DR

This work tackles catastrophic forgetting in heterogeneous federated learning for HAR in pervasive computing environments. It introduces FlexFed, which combines offline client training and performance-adaptive memory allocation to preserve knowledge across non-IID, intermittently available data streams, and employs a novel forgetting metric $F^r$ within a HAR-oriented evaluation framework. Empirical results show that FlexFed achieves $10$–$15\%$ efficiency gains and faster, more stable convergence, with especially strong improvements for infrequent or underrepresented classes. The approach enhances FL practicality for resource-constrained HAR deployments by improving memory efficiency and resilience to distribution shifts.

Abstract

Federated Learning (FL) enables collaborative model training while preserving privacy by allowing clients to share model updates instead of raw data. Pervasive computing environments (e.g., for Human Activity Recognition, HAR), which we focus on in this paper, are characterized by resource-constrained end devices, streaming sensor data and intermittent client participation. Variations in user behavior, common in HAR environments, often result in non-stationary data distributions. As such, existing FL approaches face challenges in HAR settings due to differing assumptions. The combined effects of HAR characteristics, namely heterogeneous data and intermittent participation, can lead to a severe issue called catastrophic forgetting (CF). Unlike Continuous Learning (CL), which addresses CF using memory and replay mechanisms, FL's privacy constraints prohibit such strategies. To tackle CF in HAR environments, we propose FlexFed, a novel FL approach that prioritizes data retention for efficient memory use and dynamically adjusts offline training frequency based on distribution shifts, client capability and offline duration. To better quantify CF in FL, we introduce a new metric that accounts for under-represented data, enabling more accurate evaluations. We also develop a realistic HAR-based evaluation framework that simulates streaming data, dynamic distributions, imbalances and varying availability. Experiments show that FlexFed mitigates CF more effectively, improves FL efficiency by 10 to 15 % and achieves faster, more stable convergence, especially for infrequent or under-represented data.

FlexFed: Mitigating Catastrophic Forgetting in Heterogeneous Federated Learning in Pervasive Computing Environments

TL;DR

This work tackles catastrophic forgetting in heterogeneous federated learning for HAR in pervasive computing environments. It introduces FlexFed, which combines offline client training and performance-adaptive memory allocation to preserve knowledge across non-IID, intermittently available data streams, and employs a novel forgetting metric within a HAR-oriented evaluation framework. Empirical results show that FlexFed achieves – efficiency gains and faster, more stable convergence, with especially strong improvements for infrequent or underrepresented classes. The approach enhances FL practicality for resource-constrained HAR deployments by improving memory efficiency and resilience to distribution shifts.

Abstract

Federated Learning (FL) enables collaborative model training while preserving privacy by allowing clients to share model updates instead of raw data. Pervasive computing environments (e.g., for Human Activity Recognition, HAR), which we focus on in this paper, are characterized by resource-constrained end devices, streaming sensor data and intermittent client participation. Variations in user behavior, common in HAR environments, often result in non-stationary data distributions. As such, existing FL approaches face challenges in HAR settings due to differing assumptions. The combined effects of HAR characteristics, namely heterogeneous data and intermittent participation, can lead to a severe issue called catastrophic forgetting (CF). Unlike Continuous Learning (CL), which addresses CF using memory and replay mechanisms, FL's privacy constraints prohibit such strategies. To tackle CF in HAR environments, we propose FlexFed, a novel FL approach that prioritizes data retention for efficient memory use and dynamically adjusts offline training frequency based on distribution shifts, client capability and offline duration. To better quantify CF in FL, we introduce a new metric that accounts for under-represented data, enabling more accurate evaluations. We also develop a realistic HAR-based evaluation framework that simulates streaming data, dynamic distributions, imbalances and varying availability. Experiments show that FlexFed mitigates CF more effectively, improves FL efficiency by 10 to 15 % and achieves faster, more stable convergence, especially for infrequent or under-represented data.
Paper Structure (14 sections, 18 equations, 8 figures, 3 tables)

This paper contains 14 sections, 18 equations, 8 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Federated Learning (FL)
  4. Forgetting in FL
  5. Role of Intermittent Availability and On-Device Storage Constraints in Forgetting During Federated Training
  6. Related Work
  7. Forgetting Formalisation
  8. FlexFed: A Methodology to Mitigate Forgetting
  9. Offline Client Learning via Opportunistic Resource-Aware Scheduling
  10. Performance-Adaptive Memory Allocation
  11. FlexFed Training Process Workflow
  12. Experimental Setup
  13. Results
  14. Conclusion

Figures (8)

  • Figure 1: Overview of Human Activity Recognition (HAR) in a FL Setting.
  • Figure 2: Impact of (a) Intermittent Availability and (b) Memory Constraints on FL Performance.
  • Figure 3: Procedures of Client k's Training Throughout a Day.
  • Figure 4: Generation of HAR Datasets idrees2023framework.
  • Figure 5: Average Test Accuracy for FlexFed, REFL and MIFA. The Shaded Regions Represent the Standard Deviation in Performance Across Various HAR Model.
  • ...and 3 more figures