LEAF: Language-EEG Aligned Foundation Model for Brain-Computer Interfaces

Abstract

Recent advances in electroencephalography (EEG) foundation models, which capture transferable EEG representations, have greatly accelerated the development of brain-computer interfaces (BCIs). However, existing approaches still struggle to incorporate language instructions as prior constraints for EEG representation learning, limiting their ability to leverage the semantic knowledge inherent in language to unify different labels and tasks. To address this challenge, we present LEAF, a foundation model for EEG--Language Alignment with Semantic Task Instruction and Querying. LEAF integrates task-aware semantic guidance to produce structured and linguistically aligned EEG embeddings, thereby enhancing decoding robustness and transferability. In the pretraining stage, we introduce a joint Spectral--Temporal Reconstruction (STR) framework that captures the coupled spectral rhythms and temporal dynamics of EEG signals. STR applies randomized spectral perturbation to enhance frequency robustness and uses two complementary temporal objectives to learn both contextual and sequential structure. In the EEG-Language alignment stage, we propose the Instruction-conditioned Q-Former (IQF). This query-based cross-attention transformer injects instruction embeddings into EEG tokens and achieves semantic alignment with textual label embeddings through learnable queries. We evaluate LEAF on 16 downstream datasets spanning motor imagery, emotion recognition, steady-state visual evoked potentials, covert speech, and healthcare tasks. LEAF achieves state-of-the-art performance on 12 of the 16 datasets and obtains the best average results across all five task categories. Importantly, our analyses reveal for the first time that explicit task instructions serve as semantic priors guiding EEG embeddings into coherent and linguistically grounded spaces. The code and pre-trained weights will be released.

Figures (9)

• Figure 1: The architecture design of LEAF. (a) joint Spectral-Temporal Reconstruction module (STR) for self-supervised EEG pretraining, combining frequency masking, global context modeling, and temporal sequence learning. (b) During multi-task instruction tuning, an Instruction-conditioned Q-Former (IQF) aligns EEG signals with language by injecting instruction embeddings and leveraging query-based cross-attention.
• Figure 2: Average balanced accuracy (%) per paradigm group for all models. LEAF consistently achieves the highest accuracy across motor imagery, emotion recognition, and the remaining paradigms (SSVEP, covert speech, mental workload).
• Figure 3: Per-dataset balanced accuracy under three instruction conditions in direct inference: no instruction, task-only instruction, and task-plus-target instruction. Across most datasets, richer instructions yield consistent gains, with the strongest improvements appearing in several motor imagery and emotion-recognition benchmarks.
• Figure 4: Effect of instruction conditioning on EEG representations. A: intra-class variance. B: inter-class distance. Richer instructions reduce intra-class variance and increase inter-class distance. C: KDE-based embedding distributions. Without instructions, class-conditional embeddings overlap substantially, whereas task and target instructions yield more compact, better separated clusters aligned with their textual prototypes.
• Figure 5: PCA visualization of Q-Former query evolution on four representative EEG decoding tasks. Queries diverge from their shared initialization and gradually form task-specific configurations across layers.