ALFEE: Adaptive Large Foundation Model for EEG Representation

TL;DR

ALFEE tackles the challenge of learning robust EEG representations with variable channel configurations by introducing a hybrid attention transformer that decouples channel-wise aggregation from temporal dynamics. It combines a channel encoder, temporal encoder, and EEG decoder within an encoder–decoder framework and trains with multi-task pretraining (GPT-based forecasting, temporal/channel masked autoencoding, and task-token classification) followed by multi-task fine-tuning using a task-specific CLS token dictionary. Across 15 pretraining datasets and six downstream tasks, ALFEE demonstrates superior generalization and transfer, outperforming state-of-the-art EEG foundation models and supervised baselines, with scaling laws suggesting larger models yield better performance. The approach offers a scalable, adaptable foundation for EEG analysis and multi-task brain-computer interface applications, while paving the way toward multimodal integrations and broader healthcare impact.

Abstract

While foundation models excel in text, image, and video domains, the critical biological signals, particularly electroencephalography(EEG), remain underexplored. EEG benefits neurological research with its high temporal resolution, operational practicality, and safety profile. However, low signal-to-noise ratio, inter-subject variability, and cross-paradigm differences hinder the generalization of current models. Existing methods often employ simplified strategies, such as a single loss function or a channel-temporal joint representation module, and suffer from a domain gap between pretraining and evaluation tasks that compromises efficiency and adaptability. To address these limitations, we propose the Adaptive Large Foundation model for EEG signal representation(ALFEE) framework, a novel hybrid transformer architecture with two learning stages for robust EEG representation learning. ALFEE employs a hybrid attention that separates channel-wise feature aggregation from temporal dynamics modeling, enabling robust EEG representation with variable channel configurations. A channel encoder adaptively compresses variable channel information, a temporal encoder captures task-guided evolution, and a hybrid decoder reconstructs signals in both temporal and frequency domains. During pretraining, ALFEE optimizes task prediction, channel and temporal mask reconstruction, and temporal forecasting to enhance multi-scale and multi-channel representation. During fine-tuning, a full-model adaptation with a task-specific token dictionary and a cross-attention layer boosts performance across multiple tasks. After 25,000 hours of pretraining, extensive experimental results on six downstream EEG tasks demonstrate the superior performance of ALFEE over existing models. Our ALFEE framework establishes a scalable foundation for biological signal analysis with implementation at https://github.com/xw1216/ALFEE.

Figures (8)

• Figure 1: Balanced accuracy performance on six datasets.
• Figure 2: The overall architecture of ALFEE. (1) Feature Extractor: processes raw EEG signals via multi-scale convolution and PSD analysis; (2) Channel Encoder: captures channel-wise dependencies using cross-attention mechanisms; (3) Temporal Encoder: processes temporal feature through stacked transformer layers, guided by learnable task token; (4) EEG Decoder: reconstructs signals through attention-based refinement; (5) Pretraining Head: learns signal reconstruction via complicated loss minimization; and (6) Finetuning Head: gather information by task-specific token for downstream tasks. All the self- and cross-attention layers exploits the multi-head attention and different attention masking strategies are applied in each module.
• Figure 3: Attention masks in a) Channel Encoder and b) EEG Decoder. Attention is enabled as yellow. Rows and columns correspond to query (Q) and key (K) in attention calculation.
• Figure 4: Ablation Study. We study the effect of four loss functions ($\mathrm{w/o}$$\mathcal{L}_{\mathrm{MAE}}$, $\mathrm{w/o}$$\mathcal{L}_{\mathrm{GPT}}$, and $\mathrm{w/o}$$\mathcal{L}_{\mathrm{DT}}$, $\mathrm{w/o}$$\mathcal{L}_{recon}$), and two feature extraction module (the frequency domain feature, $\mathrm{w/o}$$\mathrm{PSD}$ and the multi-scale convolution, $\mathrm{w/o}$$\mathrm{MSCov}$).
• Figure 5: Average Grad-CAM visualization showing the model's region of interest for target class prediction. Warmer colors indicating higher relevance, generated by computing gradient flow of channel encoder on ALFEE-B.