Simultaneous Control of Human Hand Joint Positions and Grip Force via HD-EMG and Deep Learning

TL;DR

This paper investigated three individual dynamic hand movements while applying forces in 10% and 30% of the maximum voluntary contraction while applying forces in 10% and 30% of the maximum voluntary contraction (MVC), and demonstrated significant accuracy in estimating kinetics and kinematics.

Abstract

In myoelectric control, simultaneous control of multiple degrees of freedom can be challenging due to the dexterity of the human hand. Numerous studies have focused on hand functionality, however, they only focused on a few degrees of freedom. In this paper, a 3DCNN-MLP model is proposed that uses high-density sEMG signals to estimate 20 hand joint positions and grip force simultaneously. The deep learning model maps the muscle activity to the hand kinematics and kinetics. The proposed models' performance is also evaluated in estimating grip forces with real-time resolution. This paper investigated three individual dynamic hand movements (2pinch, 3pinch, and fist closing and opening) while applying forces in 10% and 30% of the maximum voluntary contraction (MVC). The results demonstrated significant accuracy in estimating kinetics and kinematics. The average Euclidean distance across all joints and subjects was 11.01 $\pm$ 2.22 mm and the mean absolute error for offline and real-time force estimation were found to be 0.8 $\pm$ 0.33 N and 2.09 $\pm$ 0.9 N respectively. The results demonstrate that by leveraging high-density sEMG and deep learning, it is possible to estimate human hand dynamics (kinematics and kinetics), which is a step forward to practical prosthetic hands.

Figures (4)

• Figure 1: Overview of the study. \ref{['fig:a']} Neuromuscular pathway showing the flow of signals from the brain through the spinal cord to muscles. \ref{['fig:b']} Experiment setup. The participant is following the hand videos while reaching the predefined force level. \ref{['fig:c']} Three EMG grids are placed around the arm under the elbow and two grids proximal to the distal ulnar head. \ref{['fig:d']} Three hand movements performed during the experiment, 2pinch, 3pinch and grasp. \ref{['fig:e']} Deep learning model architecture for predicting hand joint positions and forces.
• Figure 2: The proposed model performance. \ref{['fig:a']} Heatmaps of error metrics for kinematics and force estimation across all subjects and postures: Kinematic errors represented by Euclidean Distance and PCC, averaged over 20 joints and 3 axes; Force estimation errors represented by MAE and PCC (10 and 30 represent the relative force level). \ref{['fig:b']} Index fingertip movement prediction across different postures for subject 3 (joint no. 5). \ref{['fig:c']} The proposed model's kinematics prediction performance compared to the steady prediction of rest position. The kinematics prediction accuracy is significantly different from the steady rest position (Student's t-test, * p $<$ 0.05).
• Figure 3: Force estimation results for subject 1 across various postures and force levels. The 10% and 30% values indicate force levels as a percentage of MVC. \ref{['fig:a']} Offline force estimation results. \ref{['fig:b']} Real-time force estimation results.
• Figure 4: Model performance across different force levels. \ref{['fig:a']} Force estimation performance for different tasks across different Force levels. Student's t-test statistical analysis is done to compare the groups. There is a significant difference between the predictions for 10% and 30% of MVC. \ref{['fig:b']} Kinematics prediction results across different force levels. There is no significant difference between the model performance for kinematics across different force values.