Context Informed Incremental Learning Improves Myoelectric Control Performance in Virtual Reality Object Manipulation Tasks

TL;DR

EMG gesture recognition systems often underperform in real-world, goal-directed tasks due to contextual shifts not captured during offline calibration. The authors implement Context Informed Incremental Learning (CIIL) in a VR object-manipulation setup to continuously adapt a Conv1D-based EMG classifier using contextual cues and pseudo-labeling, updating every two seconds. Results show CIIL reduces perceived workload and improves task efficiency, despite a modest drop in offline accuracy and some gesture-specific declines, suggesting real-time adaptation bridges the gap between lab performance and real-world usability. The work demonstrates the potential of context-aware online adaptation for practical myoelectric interfaces and provides a public codebase for further development.

Abstract

Electromyography (EMG)-based gesture recognition is a promising approach for designing intuitive human-computer interfaces. However, while these systems typically perform well in controlled laboratory settings, their usability in real-world applications is compromised by declining performance during real-time control. This decline is largely due to goal-directed behaviors that are not captured in static, offline scenarios. To address this issue, we use \textit{Context Informed Incremental Learning} (CIIL) - marking its first deployment in an object-manipulation scenario - to continuously adapt the classifier using contextual cues. Nine participants without upper limb differences completed a functional task in a virtual reality (VR) environment involving transporting objects with life-like grips. We compared two scenarios: one where the classifier was adapted in real-time using contextual information, and the other using a traditional open-loop approach without adaptation. The CIIL-based approach not only enhanced task success rates and efficiency, but also reduced the perceived workload by 7.1 %, despite causing a 5.8 % reduction in offline classification accuracy. This study highlights the potential of real-time contextualized adaptation to enhance user experience and usability of EMG-based systems for practical, goal-oriented applications, crucial elements towards their long-term adoption. The source code for this study is available at: https://github.com/BiomedicalITS/ciil-emg-vr.

Figures (6)

• Figure 1: The 8-channel (2000 Hz) SiFiBand from SiFi Labs Inc. used in this study (left), along with the corresponding raw EMG waveforms (right) recorded over 4 seconds during wrist flexion, wrist extension, and a power grip gesture.
• Figure 2: (a) Set of gestures included in the experiment and their associated VR game color and hand model. (b) Set of items to move from one table of another during the VR task and their corresponding valid grasp gestures.
• Figure 3: Interactive VR environment of this study. Throughout the trials, instructions and a timer were displayed to guide subjects.
• Figure 4: Offline testing confusion matrices. (a) Initial test, (b) post-NA test and (c) post-CIIL test.
• Figure 5: Offline classification accuracy results of the Initial, post-NA and post-CIIL testing sessions.