EMGBench: Benchmarking Out-of-Distribution Generalization and Adaptation for Electromyography

TL;DR

This paper introduces the first generalization and adaptation benchmark using machine learning for evaluating out-of-distribution performance of electromyography (EMG) classification algorithms, featuring a novel, easy-to-wear high-density EMG wearable for data collection.

Abstract

This paper introduces the first generalization and adaptation benchmark using machine learning for evaluating out-of-distribution performance of electromyography (EMG) classification algorithms. The ability of an EMG classifier to handle inputs drawn from a different distribution than the training distribution is critical for real-world deployment as a control interface. By predicting the user's intended gesture using EMG signals, we can create a wearable solution to control assistive technologies, such as computers, prosthetics, and mobile manipulator robots. This new out-of-distribution benchmark consists of two major tasks that have utility for building robust and adaptable control interfaces: 1) intersubject classification and 2) adaptation using train-test splits for time-series. This benchmark spans nine datasets--the largest collection of EMG datasets in a benchmark. Among these, a new dataset is introduced, featuring a novel, easy-to-wear high-density EMG wearable for data collection. The lack of open-source benchmarks has made comparing accuracy results between papers challenging for the EMG research community. This new benchmark provides researchers with a valuable resource for analyzing practical measures of out-of-distribution performance for EMG datasets. Our code and data from our new dataset can be found at emgbench.github.io.

Figures (3)

• Figure 1: Electrode configurations for various datasets and generalization task. (Left) We show the main categories of electrode configurations used: dry electrodes and wet electrodes. These categories can be further separated into individually placed electrodes and electrode arrays. (Middle) The electrode configurations and placements used are shown for each dataset. We include nine total EMG datasets. Ninapro DB3 includes subjects with transradial amputations (TA). (Right) We show that voltage signals from the arm are detected using EMG sensors and illustrate that gesture-specific patterns of muscle activity can occur.
• Figure 2: Training pipeline for testing generalization and adaptation. We show the training and evaluation pipeline for our benchmarking script to evaluate generalization across subjects and over time. For leave-one-subject-out cross validation (LOSO-CV), the training set involves all subjects other than the test subject, with the left-out subject $i$ changing from $1$ to $n$ between training runs. In addition, we test for adaptation for time-series, where data from the beginning of subject $i$ is used for fine-tuning after pretraining a model on data from other subjects. In our benchmark, both few-shot fine-tuning and intersession fine-tuning are used to evaluate adaptation for time-series.
• Figure 3: Varying preprocessing methods for generating heatmaps. Samples from different preprocessing methods are shown for the 128 electrode CapgMyo dataset, the 12 electrode NinaPro DB2, and the 8 electrode Myo Dataset. Values on the heatmaps correspond to the index of the electrode for the sub-image shown on the grid. All samples correspond to the closed-hand gesture. The closed-hand gesture primarily activates deep muscles of the flexor side of the forearm, such as the flexor digitorum superficialis.