Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MOTION: ML-Assisted On-Device Low-Latency Motion Recognition

Veeramani Pugazhenthi, Wei-Hsiang Chu, Junwei Lu, Jadyn N. Miyahira, Soheil Salehi

TL;DR

This work tackles on-device, low-latency gesture recognition for medical wearables using only tri-axial accelerometer data. It introduces an AutoML-driven framework (Piccolo AI) to extract informative features and compares four lightweight classifiers deployed on the WeBe Band, with Neural Networks delivering the best balance of accuracy and latency. Across a 60/20/20 train/validation/test split, all approaches exceed 95% accuracy, and the Neural Network achieves sub-2 ms inference times, highlighting the viability of private, on-device gesture recognition for medical monitoring. Data from five participants performing 'X', 'O', and random gestures at 25 Hz, augmented for robustness, underpins the approach and demonstrates practical applicability in wearables. The results motivate further integration of additional biomarkers and energy-efficient triggering to enable always-on operation in medical settings.

Abstract

The use of tiny devices capable of low-latency gesture recognition is gaining momentum in everyday human-computer interaction and especially in medical monitoring fields. Embedded solutions such as fall detection, rehabilitation tracking, and patient supervision require fast and efficient tracking of movements while avoiding unwanted false alarms. This study presents an efficient solution on how to build very efficient motion-based models only using triaxial accelerometer sensors. We explore the capability of the AutoML pipelines to extract the most important features from the data segments. This approach also involves training multiple lightweight machine learning algorithms using the extracted features. We use WeBe Band, a multi-sensor wearable device that is equipped with a powerful enough MCU to effectively perform gesture recognition entirely on the device. Of the models explored, we found that the neural network provided the best balance between accuracy, latency, and memory use. Our results also demonstrate that reliable real-time gesture recognition can be achieved in WeBe Band, with great potential for real-time medical monitoring solutions that require a secure and fast response time.

MOTION: ML-Assisted On-Device Low-Latency Motion Recognition

TL;DR

This work tackles on-device, low-latency gesture recognition for medical wearables using only tri-axial accelerometer data. It introduces an AutoML-driven framework (Piccolo AI) to extract informative features and compares four lightweight classifiers deployed on the WeBe Band, with Neural Networks delivering the best balance of accuracy and latency. Across a 60/20/20 train/validation/test split, all approaches exceed 95% accuracy, and the Neural Network achieves sub-2 ms inference times, highlighting the viability of private, on-device gesture recognition for medical monitoring. Data from five participants performing 'X', 'O', and random gestures at 25 Hz, augmented for robustness, underpins the approach and demonstrates practical applicability in wearables. The results motivate further integration of additional biomarkers and energy-efficient triggering to enable always-on operation in medical settings.

Abstract

The use of tiny devices capable of low-latency gesture recognition is gaining momentum in everyday human-computer interaction and especially in medical monitoring fields. Embedded solutions such as fall detection, rehabilitation tracking, and patient supervision require fast and efficient tracking of movements while avoiding unwanted false alarms. This study presents an efficient solution on how to build very efficient motion-based models only using triaxial accelerometer sensors. We explore the capability of the AutoML pipelines to extract the most important features from the data segments. This approach also involves training multiple lightweight machine learning algorithms using the extracted features. We use WeBe Band, a multi-sensor wearable device that is equipped with a powerful enough MCU to effectively perform gesture recognition entirely on the device. Of the models explored, we found that the neural network provided the best balance between accuracy, latency, and memory use. Our results also demonstrate that reliable real-time gesture recognition can be achieved in WeBe Band, with great potential for real-time medical monitoring solutions that require a secure and fast response time.

Paper Structure

This paper contains 13 sections, 3 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. Data
  4. Data Collection
  5. Data Annotation
  6. Data Augmentation
  7. ML Model Exploration
  8. AutoML
  9. Feature Selection
  10. Model Generation
  11. Summary of Results
  12. Model Profiling on the WeBe Device
  13. Conclusion and Future Work

Figures (3)

  • Figure 1: Illustration of the "X" and "O" gestures from the perspective of an external observer. The numbers indicate the sequential movement steps performed by the participant.
  • Figure 2: Representative tri-axial accelerometer signals for the three gesture classes, highlighted in color. From top to bottom (a) "X," (b) "O," and (c) Random Gestures. Each panel shows two instances of 4-sec gestures (highlighted in color), exhibiting the characteristic motion patterns that distinguish the classes.
  • Figure 3: t-SNE visualization of the selected feature clusters for different gesture classes on the WeBe Band dataset.