Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multi-Frequency Federated Learning for Human Activity Recognition Using Head-Worn Sensors

Dario Fenoglio, Mohan Li, Davide Casnici, Matias Laporte, Shkurta Gashi, Silvia Santini, Martin Gjoreski, Marc Langheinrich

TL;DR

This work tackles privacy concerns in HAR by enabling joint model learning across devices with different sensor frequencies through multi-frequency Federated Learning on head-worn devices. It adapts STResNet to a federated, frequency-heterogeneous setting and evaluates on two head-worn HAR datasets (USI-HEAR and OCOsense). Results show that multi-frequency FL achieves competitive performance with centralized training and outperforms single-frequency baselines, while accommodating asynchronous participation and heterogeneous hardware. The study highlights the practical potential of privacy-preserving HAR for earables and glasses, and provides a public implementation to spur further research.

Abstract

Human Activity Recognition (HAR) benefits various application domains, including health and elderly care. Traditional HAR involves constructing pipelines reliant on centralized user data, which can pose privacy concerns as they necessitate the uploading of user data to a centralized server. This work proposes multi-frequency Federated Learning (FL) to enable: (1) privacy-aware ML; (2) joint ML model learning across devices with varying sampling frequency. We focus on head-worn devices (e.g., earbuds and smart glasses), a relatively unexplored domain compared to traditional smartwatch- or smartphone-based HAR. Results have shown improvements on two datasets against frequency-specific approaches, indicating a promising future in the multi-frequency FL-HAR task. The proposed network's implementation is publicly available for further research and development.

Multi-Frequency Federated Learning for Human Activity Recognition Using Head-Worn Sensors

TL;DR

This work tackles privacy concerns in HAR by enabling joint model learning across devices with different sensor frequencies through multi-frequency Federated Learning on head-worn devices. It adapts STResNet to a federated, frequency-heterogeneous setting and evaluates on two head-worn HAR datasets (USI-HEAR and OCOsense). Results show that multi-frequency FL achieves competitive performance with centralized training and outperforms single-frequency baselines, while accommodating asynchronous participation and heterogeneous hardware. The study highlights the practical potential of privacy-preserving HAR for earables and glasses, and provides a public implementation to spur further research.

Abstract

Human Activity Recognition (HAR) benefits various application domains, including health and elderly care. Traditional HAR involves constructing pipelines reliant on centralized user data, which can pose privacy concerns as they necessitate the uploading of user data to a centralized server. This work proposes multi-frequency Federated Learning (FL) to enable: (1) privacy-aware ML; (2) joint ML model learning across devices with varying sampling frequency. We focus on head-worn devices (e.g., earbuds and smart glasses), a relatively unexplored domain compared to traditional smartwatch- or smartphone-based HAR. Results have shown improvements on two datasets against frequency-specific approaches, indicating a promising future in the multi-frequency FL-HAR task. The proposed network's implementation is publicly available for further research and development.

Paper Structure

This paper contains 24 sections, 2 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Federated Learning
  4. Human Activity Recognition
  5. Federated Learning for Human Activity Recognition
  6. Datasets
  7. USI-HEAR Dataset
  8. OCOsense Smart Glasses HAR Dataset
  9. Methods
  10. Centralized Machine Learning
  11. Federated Learning
  12. Multi-Frequency STResNet
  13. Experiments and Results
  14. Model Selection with Centralized Training
  15. Comparison between Centralized and Federated Learning
  16. ...and 9 more sections

Figures (2)

  • Figure 1: Encoder architecture for a single input channel. The input undergoes both a temporal and a spectral encoding to generate the latent representation.
  • Figure 2: Multi-frequency model architecture designed for devices in low- and full-battery modes. A context vector is combined with the concatenation of the encoders' outputs to zero-out any absent sensors. Finally, fully connected layers produce HAR predictions from the combined latent representation.