ChatEMG: Synthetic Data Generation to Control a Robotic Hand Orthosis for Stroke

TL;DR

This paper tackles the data bottleneck and variability in EMG-based intent inferral for stroke-hand orthoses. It introduces ChatEMG, a transformer-based autoregressive model trained on a large offline EMG corpus that can generate personalized synthetic EMG sequences conditioned on prompts from a new condition/session/subject. By augmenting a small amount of real data with synthetic samples, ChatEMG improves classifier performance across multiple architectures and adaptation scenarios, and can be deployed within a single patient session to enable functional tasks. The work demonstrates substantial gains in intent inferral accuracy and provides visualization evidence that synthetic data closely matches real data distributions, supporting practical deployment in rehabilitation robotics.

Abstract

Intent inferral on a hand orthosis for stroke patients is challenging due to the difficulty of data collection. Additionally, EMG signals exhibit significant variations across different conditions, sessions, and subjects, making it hard for classifiers to generalize. Traditional approaches require a large labeled dataset from the new condition, session, or subject to train intent classifiers; however, this data collection process is burdensome and time-consuming. In this paper, we propose ChatEMG, an autoregressive generative model that can generate synthetic EMG signals conditioned on prompts (i.e., a given sequence of EMG signals). ChatEMG enables us to collect only a small dataset from the new condition, session, or subject and expand it with synthetic samples conditioned on prompts from this new context. ChatEMG leverages a vast repository of previous data via generative training while still remaining context-specific via prompting. Our experiments show that these synthetic samples are classifier-agnostic and can improve intent inferral accuracy for different types of classifiers. We demonstrate that our complete approach can be integrated into a single patient session, including the use of the classifier for functional orthosis-assisted tasks. To the best of our knowledge, this is the first time an intent classifier trained partially on synthetic data has been deployed for functional control of an orthosis by a stroke survivor. Videos, source code, and additional information can be found at https://jxu.ai/chatemg.

Figures (6)

• Figure 1: Approach overview. Our hand orthosis (top row) collects EMG data from a forearm armband and uses this data to infer the patient's intent. ChatEMG models trained on a large corpus of offline data can generate synthetic data (middle row) for a new patient conditioned on a prompt from a small dataset of the new patient, and specific to an intended arm movement. The synthetic and real data are then used to jointly train an intent classifier, which, in the course of the same session, enables functional pick-and-place tasks (bottom row) with the orthosis.
• Figure 2: ChatEMG overview. Stage 1: ChatEMG is trained on large offline data from different conditions, sessions, and subjects. We visualize the EMG recordings from different conditions of the same subject in a single session. As shown here, there is a drastic variation in EMG signals for different conditions. Stage 2: we only need a very limited labeled dataset from a new condition, session, and subject, and we use ChatEMG to expand this limited dataset with synthetic samples. These synthetic samples are conditioned on prompts from the new condition/session/subject. Stage 3: we train intent inferral classifiers using both the synthetic samples and the original limited dataset. Running the classifier, our orthosis can then provide active assistance for the stroke subjects in functional tasks.
• Figure 3: ChatEMG model architecture. ChatEMG has two branches: the self branch that takes in the first channel (C1) and the context branch that takes in all 8-channel EMG signals.
• Figure 4: Autoregressive synthetic data generation. ChatEMG only predicts the next EMG value for the first channel (C1) and in order to generate the complete 8-channel EMG signal, we rotate the input signals 7 times such that all other channels will become the first channel once.
• Figure 5: Comparison between the real and synthetic samples on open and close intents of subjects S1, S2, and S3. The vertical line indicates the switch from the provided prompt to the generated synthetic sequence. These samples also demonstrate the significant variations in EMG signals across different stroke subjects.