NeurIPT: Foundation Model for Neural Interfaces

TL;DR

NeurIPT tackles the challenge of generalizing EEG decoding across subjects, tasks, and recording configurations by integrating a 3D spatial embedding, amplitude-aware masked pretraining, and a progressive mixture-of-experts architecture, followed by region-aware fine-tuning. The framework leverages a Crossformer TSA backbone to learn spatio-temporal EEG representations and employs Intra-Inter Lobe Pooling to exploit regional brain activity during downstream tasks. Across eight diverse EEG BCIs, NeurIPT achieves state-of-the-art results and demonstrates robust generalization, efficiency, and scalability, underpinning its potential as a foundation for universal neural decoding in EEG. The work provides actionable design choices for building scalable EEG foundation models, with thoughtful ablations and analyses that illuminate the roles of spatial encoding, targeted masking, and adaptive expert routing in handling EEG heterogeneity.

Abstract

Electroencephalography (EEG) has wide-ranging applications, from clinical diagnosis to brain-computer interfaces (BCIs). With the increasing volume and variety of EEG data, there has been growing interest in establishing foundation models (FMs) to scale up and generalize neural decoding. Despite showing early potential, applying FMs to EEG remains challenging due to substantial inter-subject, inter-task, and inter-condition variability, as well as diverse electrode configurations across recording setups. To tackle these open challenges, we propose NeurIPT, a foundation model developed for diverse EEG-based Neural Interfaces with a Pre-trained Transformer by capturing both homogeneous and heterogeneous spatio-temporal characteristics inherent in EEG signals. Temporally, we introduce Amplitude-Aware Masked Pretraining (AAMP), masking based on signal amplitude rather than random intervals, to learn robust representations across varying signal intensities beyond local interpolation. Moreover, this temporal representation is enhanced by a Progressive Mixture-of-Experts (PMoE) architecture, where specialized expert subnetworks are progressively introduced at deeper layers, adapting effectively to the diverse temporal characteristics of EEG signals. Spatially, NeurIPT leverages the 3D physical coordinates of electrodes, enabling effective transfer of embedding across varying EEG settings, and develops Intra-Inter Lobe Pooling (IILP) during fine-tuning to efficiently exploit regional brain features. Empirical evaluations across eight downstream BCI datasets, via fine-tuning, demonstrated NeurIPT consistently achieved state-of-the-art performance, highlighting its broad applicability and robust generalization. Our work pushes forward the state of FMs in EEG and offers insights into scalable and generalizable neural information processing systems.

Figures (28)

• Figure 1: Spectrograms of the nine EEG datasets reveal both homogeneous and heterogeneous spectral patterns, with some datasets showing higher power spectral density (PSD) in specific EEG frequency bands. Thus, they demand neural representations capable of adapting to input variability.
• Figure 2: BERT-like random masking v.s. our proposed amplitude-aware masking.
• Figure 3: Overview of our NeurIPT, which comprises Amplitude-Aware Masked Pretraining (AAMP), 3D Electrode Embedding, Progressive Mixture-of-Experts (PMoE), and Intra-Inter Lobe Pooling (IILP) for fine-tuning. See Figure \ref{['fig:main2']} for details on the IILP module.
• Figure 4: (Left) Intra-Inter Lobe Pooling (IILP) leverages regional brain features during fine-tuning. (Right) Visualization of attention scores from the temporal attention module and analysis of Pearson correlation between class logits and channel perturbation using Gaussian multiplicative noise. Note that colors in the right panel correspond to the brain regions depicted on the left.
• Figure 5: Models performance on various BCI downtream tasks.