Neural decoding from stereotactic EEG: accounting for electrode variability across subjects

TL;DR

This work introduces a scalable approach towards sEEG data integration for multi-subject model training, paving the way for cross-subject generalization for sEEG decoding.

Abstract

Deep learning based neural decoding from stereotactic electroencephalography (sEEG) would likely benefit from scaling up both dataset and model size. To achieve this, combining data across multiple subjects is crucial. However, in sEEG cohorts, each subject has a variable number of electrodes placed at distinct locations in their brain, solely based on clinical needs. Such heterogeneity in electrode number/placement poses a significant challenge for data integration, since there is no clear correspondence of the neural activity recorded at distinct sites between individuals. Here we introduce seegnificant: a training framework and architecture that can be used to decode behavior across subjects using sEEG data. We tokenize the neural activity within electrodes using convolutions and extract long-term temporal dependencies between tokens using self-attention in the time dimension. The 3D location of each electrode is then mixed with the tokens, followed by another self-attention in the electrode dimension to extract effective spatiotemporal neural representations. Subject-specific heads are then used for downstream decoding tasks. Using this approach, we construct a multi-subject model trained on the combined data from 21 subjects performing a behavioral task. We demonstrate that our model is able to decode the trial-wise response time of the subjects during the behavioral task solely from neural data. We also show that the neural representations learned by pretraining our model across individuals can be transferred in a few-shot manner to new subjects. This work introduces a scalable approach towards sEEG data integration for multi-subject model training, paving the way for cross-subject generalization for sEEG decoding.

Figures (7)

• Figure 1: Outline of our network architecture. The sEEG signals are converted to vector embeddings through temporal convolutions and then processed by sequential self-attention operations in the time and electrode dimensions in an alternating fashion. The latents are compressed and projected through subject-specific task heads to obtain behavioral predictions.
• Figure 2: Overview of behavioral experiment.A. Schematic of the color change detection task. B. Electrode placement projected onto the MNI brain template for four example subjects in our cohort. Red dots show electrodes used for model training; grey dots show electrodes excluded from model training (see section \ref{['sec:signal_processing']}). C. Electrodes used for model training across all subjects in our cohort projected on an MNI brain template.
• Figure 3: Comparing decoding performance between the single-subject and multi-subject models for each subject.A. Single-subject vs multi-subject model. B. Single-subject vs finetuned, multi-subject model. Circle size denotes the number of trials and color the number of electrodes (cyan to magenta represents ascending order)
• Figure 4: Comparing decoding performance between(A) Transferred single-subject finetuned models vs single-subject models trained from scratch, and (B) Transferred single-subject finetuned models vs the multi-session, multi-subject model.
• Figure 5: Decoding performance (mean $\pm$ sem) for various baselines and our proposed models.