EEG-EMG FAConformer: Frequency Aware Conv-Transformer for the fusion of EEG and EMG

TL;DR

The paper tackles motor pattern recognition by fusing EEG and EMG signals using a lightweight FAConformer architecture. It introduces a Frequency Band Attention mechanism and a suite of fusion-oriented modules (Multi-Scale Fusion, Independent Channel-Specific Convolution Module, and Fuse Module) to extract and integrate temporal and frequency information across modalities. Empirical results on the Jeong2020 dataset show state-of-the-art performance in motor execution and imagery tasks, with robust ablations confirming each component’s contribution. The work advances multimodal BCI by effectively suppressing irrelevant information and capturing multi-scale dynamics, with implications for rehabilitation-oriented interfaces and real-world applicability.

Abstract

Motor pattern recognition paradigms are the main forms of Brain-Computer Interfaces(BCI) aimed at motor function rehabilitation and are the most easily promoted applications. In recent years, many researchers have suggested encouraging patients to perform real motor control execution simultaneously in MI-based BCI rehabilitation training systems. Electromyography (EMG) signals are the most direct physiological signals that can assess the execution of movements. Multimodal signal fusion is practically significant for decoding motor patterns. Therefore, we introduce a multimodal motion pattern recognition algorithm for EEG and EMG signals: EEG-EMG FAConformer, a method with several attention modules correlated with temporal and frequency information for motor pattern recognition. We especially devise a frequency band attention module to encode EEG information accurately and efficiently. What's more, modules like Multi-Scale Fusion Module, Independent Channel-Specific Convolution Module(ICSCM), and Fuse Module which can effectively eliminate irrelevant information in EEG and EMG signals and fully exploit hidden dynamics are developed and show great effects. Extensive experiments show that EEG-EMG FAConformer surpasses existing methods on Jeong2020 dataset, showcasing outstanding performance, high robustness and impressive stability.

Figures (7)

• Figure 1: Overall framework of our network
• Figure 2: Accuracy Comparison Of Models
• Figure 3: Visualization of heads for multigrasp-mi: RAW and Features EEG-EMG FAConfomer extracts
• Figure 4: t-SNE Dimensionality Reduction Comparison of Multigrasp-mi
• Figure 5: t-SNE Dimensionality Reduction Comparison of Twist-mi