Reciprocal Learning of Intent Inferral with Augmented Visual Feedback for Stroke

TL;DR

The paper introduces reciprocal learning, a bidirectional framework in which stroke survivors and an EMG-based intent inferral classifier iteratively adapt to one another using augmented visual feedback. By interleaving data collection, classifier training, and reciprocal practice with LED-based probability displays, the approach aims to produce more separable EMG patterns and improve prediction accuracy in a wearable hand orthosis. In a preliminary study with five chronic stroke subjects, the method improved intent inferral accuracy for two participants (S1 and S4) and yielded evidence of greater data separability (via weight variance and t-SNE plots) for those individuals, while others remained unchanged. The work demonstrates potential for personalized, co-adaptive rehabilitation interfaces, while acknowledging variability across subjects and highlighting the need for longer, multi-iteration trials to fully assess efficacy and generalizability.

Abstract

Intent inferral, the process by which a robotic device predicts a user's intent from biosignals, offers an effective and intuitive way to control wearable robots. Classical intent inferral methods treat biosignal inputs as unidirectional ground truths for training machine learning models, where the internal state of the model is not directly observable by the user. In this work, we propose reciprocal learning, a bidirectional paradigm that facilitates human adaptation to an intent inferral classifier. Our paradigm consists of iterative, interwoven stages that alternate between updating machine learning models and guiding human adaptation with the use of augmented visual feedback. We demonstrate this paradigm in the context of controlling a robotic hand orthosis for stroke, where the device predicts open, close, and relax intents from electromyographic (EMG) signals and provides appropriate assistance. We use LED progress-bar displays to communicate to the user the predicted probabilities for open and close intents by the classifier. Our experiments with stroke subjects show reciprocal learning improving performance in a subset of subjects (two out of five) without negatively impacting performance on the others. We hypothesize that, during reciprocal learning, subjects can learn to reproduce more distinguishable muscle activation patterns and generate more separable biosignals.

Figures (4)

• Figure 1: Reciprocal learning with robotic hand orthosis and augmented visual feedback. Our hand orthosis assists stroke survivors in opening the hand when an open intent is detected from EMG signals, and allows the hand to close when it detects the close intent. Augmented feedback consists of LED display bars corresponding to the predicted probabilities of open and close from the intent classifier running on the orthosis. Subjects use visual feedback to adapt to the intent inferral model during reciprocal learning practice. The bottom row shows a stroke survivor using our device with augmented feedback for a functional pick-and-place task.
• Figure 2: The goal of reciprocal learning is to guide the subject in generating more separable signals. Before reciprocal learning practice, the collected data are more diverse for each intent, making it difficult for the classifier to learn a perfect decision boundary. During reciprocal learning practice, the subject adapts to the intent inferral classifier and learns to generate data in more separable regions. After reciprocal learning practice, we retrain the classifier using the newly generated more separable data. This figure is intended to illustrate the concept, and does not show EMG signals collected from real subjects; separability data from real stroke subjects will be shown in Fig. \ref{['fig:separability']}.
• Figure 3: Reciprocal learning paradigm. An iteration consists of data collection, classifier training, performance evaluation, and reciprocal learning practice.
• Figure 4: Reciprocal learning results on two stroke subjects.Top: comparing data separability. After reciprocal learning, clusters associated with each intent become more easily separable. Bottom: LDA classifier weights visualization indicating a specific EMG electrode (0 through 7) is negatively (blue) or positively (red) associated with a specific intent. Reciprocal learning strengthens such associations, which helps separability.