Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Forearm Ultrasound based Gesture Recognition on Edge

Keshav Bimbraw, Haichong K. Zhang, Bashima Islam

TL;DR

The deployment of deep neural networks for forearm ultrasound-based hand gesture recognition on edge devices using quantization techniques is explored, achieving substantial reductions in model size while maintaining high accuracy and low latency.

Abstract

Ultrasound imaging of the forearm has demonstrated significant potential for accurate hand gesture classification. Despite this progress, there has been limited focus on developing a stand-alone end- to-end gesture recognition system which makes it mobile, real-time and more user friendly. To bridge this gap, this paper explores the deployment of deep neural networks for forearm ultrasound-based hand gesture recognition on edge devices. Utilizing quantization techniques, we achieve substantial reductions in model size while maintaining high accuracy and low latency. Our best model, with Float16 quantization, achieves a test accuracy of 92% and an inference time of 0.31 seconds on a Raspberry Pi. These results demonstrate the feasibility of efficient, real-time gesture recognition on resource-limited edge devices, paving the way for wearable ultrasound-based systems.

Forearm Ultrasound based Gesture Recognition on Edge

TL;DR

The deployment of deep neural networks for forearm ultrasound-based hand gesture recognition on edge devices using quantization techniques is explored, achieving substantial reductions in model size while maintaining high accuracy and low latency.

Abstract

Ultrasound imaging of the forearm has demonstrated significant potential for accurate hand gesture classification. Despite this progress, there has been limited focus on developing a stand-alone end- to-end gesture recognition system which makes it mobile, real-time and more user friendly. To bridge this gap, this paper explores the deployment of deep neural networks for forearm ultrasound-based hand gesture recognition on edge devices. Utilizing quantization techniques, we achieve substantial reductions in model size while maintaining high accuracy and low latency. Our best model, with Float16 quantization, achieves a test accuracy of 92% and an inference time of 0.31 seconds on a Raspberry Pi. These results demonstrate the feasibility of efficient, real-time gesture recognition on resource-limited edge devices, paving the way for wearable ultrasound-based systems.
Paper Structure (16 sections, 3 figures, 1 table)

This paper contains 16 sections, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Experimental Framework and Implementation
  3. Data Collection
  4. CNN Architecture and Training
  5. System Description
  6. Model Quantization
  7. Evaluation
  8. Results and Discussion
  9. Baseline Performance Analysis
  10. Model Optimization and Deployment
  11. Model Size Reduction and Quantization
  12. Inference Latency Analysis
  13. Performance Evaluation on Raspberry Pi
  14. Demonstrations
  15. Limitations and Future Work
  16. ...and 1 more sections

Figures (3)

  • Figure 1: Schematic for ultrasound based gesture estimation deployed on a Raspberry Pi. The ultrasound image is obtained from an ultrasound probe, and it is sent to the Raspberry Pi over UDP after downsizing. Here, a pre-trained and optimized model is used for inference.
  • Figure 2: Hand gestures: (a) Index Pinch, (b) Middle Pinch, (c) Ring Pinch, and Open Hand (not shown).
  • Figure 3: Confusion matrices showing the results for baseline classification performance on the test-set for (a) 25 epochs, and for (b) 50 epochs. Table showing the results for different types of quantizations with size in bytes.