Muscle Anatomy-aware Geometric Deep Learning for sEMG-based Gesture Decoding

TL;DR

This work tackles robust gesture decoding from non-invasive sEMG signals, which are non-Euclidean and exhibit strong inter-subject/session variability. It introduces TMKNet, a geometry-aware network that learns on the SPD manifold $\mathcal{S}^{++}$ with an anatomy-informed Euclidean stem and a Riemannian backbone, enhanced by unsupervised domain adaptation via domain-specific batch normalization. The method demonstrates state-of-the-art inter-session generalization on NinaproDB6 and FlexWear-HD, outperforming Euclidean baselines and a prior SPD-based model, and provides interpretable insights into muscle-specific activations. The approach advances manifold-based learning for muscle signals and opens new research directions for robust, calibration-free sEMG gesture recognition in real-world settings, with potential extensions to test-time adaptation and inter-subject generalization.

Abstract

Robust and accurate decoding of gesture from non-invasive surface electromyography (sEMG) is important for various applications including spatial computing, healthcare, and entertainment, and has been actively pursued by researchers and industry. Majority of sEMG-based gesture decoding algorithms employ deep neural networks that are designed for Euclidean data, and may not be suitable for analyzing multi-dimensional, non-stationary time-series with long-range dependencies such as sEMG. State-of-the-art sEMG-based decoding methods also demonstrate high variability across subjects and sessions, requiring re-calibration and adaptive fine-tuning to boost performance. To address these shortcomings, this work proposes a geometric deep learning model that learns on symmetric positive definite (SPD) manifolds and leverages unsupervised domain adaptation to desensitize the model to subjects and sessions. The model captures the features in time and across sensors with multiple kernels, projects the features onto SPD manifold, learns on manifolds and projects back to Euclidean space for classification. It uses a domain-specific batch normalization layer to address variability between sessions, alleviating the need for re-calibration or fine-tuning. Experiments with publicly available benchmark gesture decoding datasets (Ninapro DB6, Flexwear-HD) demonstrate the superior generalizability of the model compared to Euclidean and other SPD-based models in the inter-session scenario, with up to 8.83 and 4.63 points improvement in accuracy, respectively. Detailed analyses reveal that the model extracts muscle-specific information for different tasks and ablation studies highlight the importance of modules introduced in the work. The proposed method pushes the state-of-the-art in sEMG-based gesture recognition and opens new research avenues for manifold-based learning for muscle signals.

Figures (5)

• Figure 1: Proposed end-to-end Riemannian network for gesture classification from sEMG. The multi-channel data is fed to the Euclidean stem for multi-scale temporal and spatial feature extraction with kernels $k_t^i$ and $k_s^j$, respectively. The features are transformed to SPD manifold, processed on manifolds and transformed back for final classification. Markers represent domains (subjects/sessions) and colours represent classes.
• Figure 2: Subject-wise inter-session classification accuracy and F1 score with TMKNet and TSMNet (a) accuracy for FlexWear-HD dataset, (b) F1 score for Flexwear-HD, (c)Accuracy for NinaproDB6 dataset, (d) F1 score for NinaproDB6 dataset
• Figure 3: Sensor-wise maximum saliency for wrist extension and flexion tasks for a sample subject and session in Flexwear-HD dataset.
• Figure 4: t-SNE of features before and after domain-specific batch normalization for Flexwear-HD dataset. The colour of the points indicates the class, while the marker indicates the domain ($\star$: target domain, Others: source domain).
• Figure 5: Confusion matrix for NinaproDB6 (subject 1, session 1). LD: Large Diameter, AT: Adducted Thumb, IFE: Index Finger Extension, MW: Medium Wrap, WT: Writing Tripod, PS: Power Sphere, PrS: Precision Sphere.