ISAM-MTL: Cross-subject multi-task learning model with identifiable spikes and associative memory networks

TL;DR

This work tackles cross-subject EEG variability that impedes cross-subject BCI performance by introducing ISAM-MTL, a multi-task learning model that combines identifiable spiking representations with a subject-specific associative memory classifier. The spike encoder uses label-guided variational inference inspired by pi-VAE to produce identifiable latent spikes, while the associative memory matrices enable fast, few-shot subject adaptation via Hebbian learning with linear time complexity $O(n)$. On two BCI datasets, ISAM-MTL achieves state-of-the-art cross-subject accuracy (e.g., $84.1\%$ with $\sigma=0.061$ on BCI IV IIa) and demonstrates strong few-shot performance (e.g., $>90\%$ with 40 samples and 5-shot for 2-class tasks on BCI III Iva), along with improved identifiability of neural activity. The approach offers rapid, interpretable calibration for BCI systems and highlights the potential of combining identifiable spike representations with memory-based learning for cross-subject EEG decoding.

Abstract

Cross-subject variability in EEG degrades performance of current deep learning models, limiting the development of brain-computer interface (BCI). This paper proposes ISAM-MTL, which is a multi-task learning (MTL) EEG classification model based on identifiable spiking (IS) representations and associative memory (AM) networks. The proposed model treats EEG classification of each subject as an independent task and leverages cross-subject data training to facilitate feature sharing across subjects. ISAM-MTL consists of a spiking feature extractor that captures shared features across subjects and a subject-specific bidirectional associative memory network that is trained by Hebbian learning for efficient and fast within-subject EEG classification. ISAM-MTL integrates learned spiking neural representations with bidirectional associative memory for cross-subject EEG classification. The model employs label-guided variational inference to construct identifiable spike representations, enhancing classification accuracy. Experimental results on two BCI Competition datasets demonstrate that ISAM-MTL improves the average accuracy of cross-subject EEG classification while reducing performance variability among subjects. The model further exhibits the characteristics of few-shot learning and identifiable neural activity beneath EEG, enabling rapid and interpretable calibration for BCI systems.

Figures (6)

• Figure 1: The architecture of the proposed ISAM-MTL model. The model proceeds with two stages. Stage 1: the cross-subject trained variational spiking encoder uses a 1D CNN and LIF neuron population to construct an autoencoder to extract low-dimensional spiking representations of EEG, and uses label-guided variational inference to enhance the identifiability of latent space spike. Stage 2: the associative memory classifier gives a AM trained within the subject using Hebbian learning for each subject, and the AM maps EEG to classes.
• Figure 2: The distribution of latent spike features after dimensionality reduction using t-SNE. (a) Variational inference without label guidance. (b) Variational inference with label guidance.
• Figure 3: Comparison of ISAM-MTL with other subject-specific classification models on the BCI Competition IV IIa dataset, (a) EEGNet32lawhern2018eegnet, (b) ATCNet33altaheri2022physics (SOTA), (c) FBMSNet34liu2022fbmsnet, (d) ISAM-MTL.
• Figure 4: Ablation experiments of our model on (I) BCI Competition IV IIa and (II) BCI Competition III Iva. (a) ISAM-MTL. (b) Remove label-guided variational inference. (c) Gradient descent fully connected layers replace associative memory matrix model. (d) Tanh activation function replaces LIF neurons.
• Figure 5: ISAM-MTL few-shot learning on the BCI Competition III Iva dataset, where $n$ is the number of samples for each class.