SpGesture: Source-Free Domain-adaptive sEMG-based Gesture Recognition with Jaccard Attentive Spiking Neural Network

TL;DR

A novel SpGesture framework based on Spiking Neural Networks, which possesses several unique merits compared with existing methods, including the introduction of Source-Free Domain Adaptation into SNN for the first time and a notable rise in system accuracy.

Abstract

Surface electromyography (sEMG) based gesture recognition offers a natural and intuitive interaction modality for wearable devices. Despite significant advancements in sEMG-based gesture-recognition models, existing methods often suffer from high computational latency and increased energy consumption. Additionally, the inherent instability of sEMG signals, combined with their sensitivity to distribution shifts in real-world settings, compromises model robustness. To tackle these challenges, we propose a novel SpGesture framework based on Spiking Neural Networks, which possesses several unique merits compared with existing methods: (1) Robustness: By utilizing membrane potential as a memory list, we pioneer the introduction of Source-Free Domain Adaptation into SNN for the first time. This enables SpGesture to mitigate the accuracy degradation caused by distribution shifts. (2) High Accuracy: With a novel Spiking Jaccard Attention, SpGesture enhances the SNNs' ability to represent sEMG features, leading to a notable rise in system accuracy. To validate SpGesture's performance, we collected a new sEMG gesture dataset which has different forearm postures, where SpGesture achieved the highest accuracy among the baselines ($89.26\%$). Moreover, the actual deployment on the CPU demonstrated a system latency below 100ms, well within real-time requirements. This impressive performance showcases SpGesture's potential to enhance the applicability of sEMG in real-world scenarios. The code is available at https://github.com/guoweiyu/SpGesture/.

Figures (11)

• Figure 1: The pipeline of Jaccard Attention Spike Neural Network: Raw sEMG Data is first encoded into Spike Signals using ConvLIF. These signals pass through ConvLIF layers with $N$ and $2N$ channels. The processed data then goes through the Spiking Jaccard Attention mechanism.
• Figure 2: Comparison of MA-SNN and Spiking Jaccard Attention Modules. MA-SNN yao2023attention uses fully connected layers with pooling but lacks a querying mechanism, leading to continuous intermediate values and lower efficiency. Our Spiking Jaccard Attention uses spike values for intermediate representations, enhancing efficiency and accuracy.
• Figure 3: Computation flow of Spiking Source-Free Domain Adaptation. The process starts with selecting the $k$-nearest samples from the membrane potential memory using the Pearson correlation coefficient. Probabilistic Label Generation then produces pseudo-labels based on these $k$ samples. Gradients are computed with Smooth NLL and KL divergence loss. The membrane potential memory list is updated at each epoch's end.
• Figure 4: Comparison of performance before and after applying SSFDA for various methodologies: Figures \ref{['fig:subfig1']} and \ref{['fig:subfig2']} are Violin Plots demonstrating this disparity.
• Figure 5: Inference speed and RAM usage comparison between spike and float data for Raw Attention vaswani2017attention, Efficient Attention shen2021efficient, and our Spiking Jaccard Attention: The first column shows inference time for float data, and the second for spike data. The third and fourth columns show RAM usage for these data types. The $x$-axis represents different data row counts, and the $y$-axis is logarithmic to highlight performance differences. Each experiment was conducted 100 times, with averaged results.