Cross-Subject Domain Adaptation for Classifying Working Memory Load with Multi-Frame EEG Images

TL;DR

This work addresses cross-subject variability in EEG-based working memory load classification. It introduces CS-DASA, a framework that converts EEG time series into multi-frame spectral-spatial images and uses a frozen Subject-Shared ConvLSTM, a MK-MMD-based transfer loss, and a subject-to-subject spatial attention mechanism to enable cross-subject generalization. On a public WM EEG dataset with 13 subjects, CS-DASA outperformed state-of-the-art baselines by substantial margins, with ablation showing the spatial attention contributes additional gains, and t-SNE visualizations illustrating better separability of loads under transfer. The approach offers a practical path to deploy WM load classifiers across new users without extensive subject-specific calibration, enhancing real-world BCI portability.

Abstract

Working memory (WM), denoting the information temporally stored in the mind, is a fundamental research topic in the field of human cognition. Electroencephalograph (EEG), which can monitor the electrical activity of the brain, has been widely used in measuring the level of WM. However, one of the critical challenges is that individual differences may cause ineffective results, especially when the established model meets an unfamiliar subject. In this work, we propose a cross-subject deep adaptation model with spatial attention (CS-DASA) to generalize the workload classifications across subjects. First, we transform EEG time series into multi-frame EEG images incorporating spatial, spectral, and temporal information. First, the Subject-Shared module in CS-DASA receives multi-frame EEG image data from both source and target subjects and learns the common feature representations. Then, in the subject-specific module, the maximum mean discrepancy is implemented to measure the domain distribution divergence in a reproducing kernel Hilbert space, which can add an effective penalty loss for domain adaptation. Additionally, the subject-to-subject spatial attention mechanism is employed to focus on the discriminative spatial features from the target image data. Experiments conducted on a public WM EEG dataset containing 13 subjects show that the proposed model is capable of achieving better performance than existing state-of-the-art methods.

Figures (7)

• Figure 1: Time course of a visual working memory task
• Figure 2: Pipeline overview of the proposed cross-subject transfer model: (1) EEG time series are divided into 7 slices of 0.5-second windows and spectral power within the three frequency bands (theta (4-7Hz), alpha (8-13Hz), and beta (13-30 Hz)) are extracted by Fast Fourier Transform (FFT); (2) A 2-D space position map of EEG electrodes formed by the azimuthal equidistant projection is then utilized to make 7-frame EEG images with three spectral channels; (3) Then, a pair of source and target EEG images are fed into the Subject-Shared module. This Subject-Shared module has been pretrained only by the source data and is frozen in the training phase; (4) Subsequently, the output feature pair is input in the Subject-Specific module. In this module, the features from different domains are further fed into the same layer architecture but with different network parameters; (5) Finally, the Class-Prediction module processes the output features by the last module and makes a decision for the level of WM load.
• Figure 3: A simplified example for an EEG node with Azimuthal Equidistant projection. $E$, holding the coordinate $\left(x,y,z\right)$, represents an electrode node from the BCI system. $E^{'}$ is the projection of $E$ on the plane $XOY$ and its coordinate is $\left(x,y,0\right)$.
• Figure 4: Different position presentations for EEG electrode nodes
• Figure 5: Visualization of features with t-SNE. Blue dots, yellow pluses, green crosses and red stars represent the latent 2-D representations of workload 1-4 after t-SNE. The first row shows the results for the task $S1 \rightarrow S2$ and the second one is the $S10 \rightarrow S12$. Each column demonstrates the results for a transfer learning method.