Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

User Training with Error Augmentation for Electromyogram-based Gesture Classification

Yunus Bicer, Niklas Smedemark-Margulies, Basak Celik, Elifnur Sunger, Ryan Orendorff, Stephanie Naufel, Tales Imbiriba, Deniz Erdoğmuş, Eugene Tunik, Mathew Yarossi

TL;DR

It is suggested that real-time feedback in a gamified user interface with manipulation of feedback may enable intuitive, rapid, and accurate task acquisition for sEMG-based gesture recognition applications.

Abstract

We designed and tested a system for real-time control of a user interface by extracting surface electromyographic (sEMG) activity from eight electrodes in a wrist-band configuration. sEMG data were streamed into a machine-learning algorithm that classified hand gestures in real-time. After an initial model calibration, participants were presented with one of three types of feedback during a human-learning stage: veridical feedback, in which predicted probabilities from the gesture classification algorithm were displayed without alteration, modified feedback, in which we applied a hidden augmentation of error to these probabilities, and no feedback. User performance was then evaluated in a series of minigames, in which subjects were required to use eight gestures to manipulate their game avatar to complete a task. Experimental results indicated that, relative to baseline, the modified feedback condition led to significantly improved accuracy and improved gesture class separation. These findings suggest that real-time feedback in a gamified user interface with manipulation of feedback may enable intuitive, rapid, and accurate task acquisition for sEMG-based gesture recognition applications.

User Training with Error Augmentation for Electromyogram-based Gesture Classification

TL;DR

It is suggested that real-time feedback in a gamified user interface with manipulation of feedback may enable intuitive, rapid, and accurate task acquisition for sEMG-based gesture recognition applications.

Abstract

We designed and tested a system for real-time control of a user interface by extracting surface electromyographic (sEMG) activity from eight electrodes in a wrist-band configuration. sEMG data were streamed into a machine-learning algorithm that classified hand gestures in real-time. After an initial model calibration, participants were presented with one of three types of feedback during a human-learning stage: veridical feedback, in which predicted probabilities from the gesture classification algorithm were displayed without alteration, modified feedback, in which we applied a hidden augmentation of error to these probabilities, and no feedback. User performance was then evaluated in a series of minigames, in which subjects were required to use eight gestures to manipulate their game avatar to complete a task. Experimental results indicated that, relative to baseline, the modified feedback condition led to significantly improved accuracy and improved gesture class separation. These findings suggest that real-time feedback in a gamified user interface with manipulation of feedback may enable intuitive, rapid, and accurate task acquisition for sEMG-based gesture recognition applications.
Paper Structure (35 sections, 10 equations, 10 figures, 1 algorithm)

This paper contains 35 sections, 10 equations, 10 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Experimental Design
  3. Subjects
  4. Experimental Setup
  5. Data Acquisition
  6. Subject Group Assignment
  7. Gesture Timing
  8. Block One: Calibration
  9. Block Two: Instructed Games
  10. Block Three: Live Feedback
  11. Block Four: Free Games
  12. Signal Modeling
  13. Feature Extraction
  14. Classification Model
  15. Modified Feedback
  16. ...and 20 more sections

Figures (10)

  • Figure 1: Electrode Placement. sEMG data is collected using $8$ Delsys Trigno sEMG sensors uniformly spaced around the right forearm.
  • Figure 2: Gesture Trial Timing. In the yellow 'prompting' epoch, the subject sees an instruction. In the green 'gesture production' epoch, the subject performs the gesture. In the red 'recovery' epoch, the subject returns to the rest position. Features for classification are extracted from the last $500$ ms of gesture production to help ensure that steady-state features are collected.
  • Figure 3: Example mini game. The blue player avatar must be moved to match the gray target avatar. The minimal path includes moving right, down twice, decreasing the die number (using a pinch gesture), and reducing size (using a fist gesture).
  • Figure 4: The participant User Interface. Top left: instructed gesture. Bottom left: predicted gesture probabilities. Right: Task window including subject's avatar and target. Outer edge: gesture epoch indicator.
  • Figure 5: Top: Real-time probability feedback window. The horizontal line at $0.5$ shows the decision threshold. Bottom: Example of probability values without modification ("Veridical") and with modification ("Modified") as described in Sec. \ref{['sec:Modified_Feedback']} for several hypothetical values of $m$. $m=0.75$ used for real experiments. Arrows highlight an example case where modification causes the gesture to become sub-threshold; participant may compensate by improving gesture quality.
  • ...and 5 more figures