Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Dendron: Enhancing Human Activity Recognition with On-Device TinyML Learning

Hazem Hesham Yousef Shalby, Manuel Roveri

TL;DR

Dendron tackles the challenge of learning new human activity recognition tasks directly on resource-limited wearable devices under scarce supervision. It achieves this with a hierarchical HAR framework that shares a convolutional feature extractor and uses task-specific fully connected heads, combined with a unified off-device training and an on-device learning process for integrating new tasks. The approach yields efficient on-device adaptation, outperforming traditional single-model and hierarchical baselines in accuracy while markedly reducing memory and computation, making on-device learning feasible on devices like STM32. The work demonstrates practical impact by enabling continual HAR learning in privacy-preserving, offline settings and points to future enhancements in dynamic hierarchy generation and drift detection.

Abstract

Human activity recognition (HAR) is a research field that employs Machine Learning (ML) techniques to identify user activities. Recent studies have prioritized the development of HAR solutions directly executed on wearable devices, enabling the on-device activity recognition. This approach is supported by the Tiny Machine Learning (TinyML) paradigm, which integrates ML within embedded devices with limited resources. However, existing approaches in the field lack in the capability for on-device learning of new HAR tasks, particularly when supervised data are scarce. To address this limitation, our paper introduces Dendron, a novel TinyML methodology designed to facilitate the on-device learning of new tasks for HAR, even in conditions of limited supervised data. Experimental results on two public-available datasets and an off-the-shelf device (STM32-NUCLEO-F401RE) show the effectiveness and efficiency of the proposed solution.

Dendron: Enhancing Human Activity Recognition with On-Device TinyML Learning

TL;DR

Dendron tackles the challenge of learning new human activity recognition tasks directly on resource-limited wearable devices under scarce supervision. It achieves this with a hierarchical HAR framework that shares a convolutional feature extractor and uses task-specific fully connected heads, combined with a unified off-device training and an on-device learning process for integrating new tasks. The approach yields efficient on-device adaptation, outperforming traditional single-model and hierarchical baselines in accuracy while markedly reducing memory and computation, making on-device learning feasible on devices like STM32. The work demonstrates practical impact by enabling continual HAR learning in privacy-preserving, offline settings and points to future enhancements in dynamic hierarchy generation and drift detection.

Abstract

Human activity recognition (HAR) is a research field that employs Machine Learning (ML) techniques to identify user activities. Recent studies have prioritized the development of HAR solutions directly executed on wearable devices, enabling the on-device activity recognition. This approach is supported by the Tiny Machine Learning (TinyML) paradigm, which integrates ML within embedded devices with limited resources. However, existing approaches in the field lack in the capability for on-device learning of new HAR tasks, particularly when supervised data are scarce. To address this limitation, our paper introduces Dendron, a novel TinyML methodology designed to facilitate the on-device learning of new tasks for HAR, even in conditions of limited supervised data. Experimental results on two public-available datasets and an off-the-shelf device (STM32-NUCLEO-F401RE) show the effectiveness and efficiency of the proposed solution.

Paper Structure

This paper contains 22 sections, 7 equations, 5 figures, 5 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related works
  3. Human Activity Recognition
  4. TinyML
  5. Proposed Methodology
  6. Formalization
  7. Proposed architecture
  8. Off-device training process
  9. On-device inference process
  10. On-Device learning process
  11. Learn the FC module $h(\cdot)^{(new)}$
  12. Update the hierarchical schema $D$
  13. Experimental results
  14. Datasets
  15. UCA-EHAR dataset
  16. ...and 7 more sections

Figures (5)

  • Figure 1: Example of hierarchical HAR schema.
  • Figure 2: Overview of the off-device training process.
  • Figure 3: Example of the process for selecting the node to add a new task $T^{(new)}$.
  • Figure 4: One-dimensional ResNetv1-6 model architecture used in the evaluation conducted in Section \ref{['sec:evaluation']}.
  • Figure 5: Number of Samples per Class for UCA-EHAR and UCI-HAPT Datasets