Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ALVI Interface: Towards Full Hand Motion Decoding for Amputees Using sEMG

Aleksandr Kovalev, Anna Makarova, Petr Chizhov, Matvey Antonov, Gleb Duplin, Vladislav Lomtev, Viacheslav Gostevskii, Vladimir Bessonov, Andrey Tsurkan, Mikhail Korobok, Aleksejs Timčenko

TL;DR

The paper addresses real-time, high-fidelity decoding of individual finger movements from sEMG in amputees. It combines a VR-based data collection platform, a transformer-based HandFormer for EMG-to-hand pose translation, and the ALVI Interface to provide adaptive, real-time control with visual feedback. Offline accuracy reaches a correlation of up to 0.86 in non-amputees and 0.80 in amputees, with 25 Hz real-time decoding and 51 ms latency, facilitated by a co-adaptive learning loop that updates weights every 10 seconds. The work demonstrates practical, personalized prosthetic control in VR and highlights directions for larger clinical trials, long-term stability, and integration with real prostheses for real-world tasks and rehabilitation.

Abstract

We present a system for decoding hand movements using surface EMG signals. The interface provides real-time (25 Hz) reconstruction of finger joint angles across 20 degrees of freedom, designed for upper limb amputees. Our offline analysis shows 0.8 correlation between predicted and actual hand movements. The system functions as an integrated pipeline with three key components: (1) a VR-based data collection platform, (2) a transformer-based model for EMG-to-motion transformation, and (3) a real-time calibration and feedback module called ALVI Interface. Using eight sEMG sensors and a VR training environment, users can control their virtual hand down to finger joint movement precision, as demonstrated in our video: youtube link.

ALVI Interface: Towards Full Hand Motion Decoding for Amputees Using sEMG

TL;DR

The paper addresses real-time, high-fidelity decoding of individual finger movements from sEMG in amputees. It combines a VR-based data collection platform, a transformer-based HandFormer for EMG-to-hand pose translation, and the ALVI Interface to provide adaptive, real-time control with visual feedback. Offline accuracy reaches a correlation of up to 0.86 in non-amputees and 0.80 in amputees, with 25 Hz real-time decoding and 51 ms latency, facilitated by a co-adaptive learning loop that updates weights every 10 seconds. The work demonstrates practical, personalized prosthetic control in VR and highlights directions for larger clinical trials, long-term stability, and integration with real prostheses for real-world tasks and rehabilitation.

Abstract

We present a system for decoding hand movements using surface EMG signals. The interface provides real-time (25 Hz) reconstruction of finger joint angles across 20 degrees of freedom, designed for upper limb amputees. Our offline analysis shows 0.8 correlation between predicted and actual hand movements. The system functions as an integrated pipeline with three key components: (1) a VR-based data collection platform, (2) a transformer-based model for EMG-to-motion transformation, and (3) a real-time calibration and feedback module called ALVI Interface. Using eight sEMG sensors and a VR training environment, users can control their virtual hand down to finger joint movement precision, as demonstrated in our video: youtube link.

Paper Structure

This paper contains 12 sections, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Methods
  3. Dataset
  4. Experiment
  5. Data preprocessing
  6. Model
  7. ALVI Interface
  8. Results
  9. Quantitative Performance
  10. Real-time Performance
  11. User Experience
  12. Discussion

Figures (3)

  • Figure 1: Data collection system. Participant with VR headset and sEMG armband performs movements with intact hand. System tracks movements, mirrors them to create virtual model of absent hand, and records muscle activity from residual limb.
  • Figure 2: Architecture of the model. The HandFormer architecture transforms muscle activity (sEMG) into hand movements through a two-stage process. The Encoder (left) tokenizes sEMG signals from 8 channels into patches and extracts relevant features. The Decoder (right) employs a Perceiver-like architecture with 32 learnable queries corresponding to predicted movement frames. This non-autoregressive design enables efficient real-time translation of muscle signals into precise finger joint angles across 20 degrees of freedom.
  • Figure 3: ALVI Interface system architecture..