BrainOmni: A Brain Foundation Model for Unified EEG and MEG Signals

TL;DR

BrainOmni addresses the lack of unified models for EEG and MEG by introducing BrainTokenizer, a spatiotemporal tokeniser with a Sensor Encoder to handle device heterogeneity, and a two-stage training pipeline that includes large-scale self-supervised pretraining on EEG and MEG data. The Criss-Cross Transformer-based BrainOmni model leverages masked token prediction to learn coherent cross-modal representations, achieving state-of-the-art performance across EEG, MEG, and EMEG downstream tasks. Key contributions include joint EEG-MEG pretraining, a sensor-geometry-aware encoding scheme, and the first quantised, spatiotemporal brain representation for unified neural signal modelling, demonstrating strong generalisation to unseen devices. The work advances scalable, cross-domain neural representation learning and paves the way for versatile brain signal analysis across diverse recording setups.

Abstract

Electroencephalography (EEG) and magnetoencephalography (MEG) measure neural activity non-invasively by capturing electromagnetic fields generated by dendritic currents. Although rooted in the same biophysics, EEG and MEG exhibit distinct signal patterns, further complicated by variations in sensor configurations across modalities and recording devices. Existing approaches typically rely on separate, modality- and dataset-specific models, which limits the performance and cross-domain scalability. This paper proposes BrainOmni, the first brain foundation model that generalises across heterogeneous EEG and MEG recordings. To unify diverse data sources, we introduce BrainTokenizer,the first tokenizer that quantises spatiotemporal brain activity into discrete representations. Central to BrainTokenizer is a novel Sensor Encoder that encodes sensor properties such as spatial layout, orientation, and type, enabling compatibility across devices and modalities. Building upon the discrete representations, BrainOmni learns unified semantic embeddings of brain signals by self-supervised pretraining. To the best of our knowledge, it is the first foundation model to support both EEG and MEG signals, as well as the first to incorporate large-scale MEG pretraining. A total of 1,997 hours of EEG and 656 hours of MEG data are curated and standardised from publicly available sources for pretraining. Experiments show that BrainOmni outperforms both existing foundation models and state-of-the-art task-specific models on a range of downstream tasks. It also demonstrates strong generalisation to unseen EEG and MEG devices. Further analysis reveals that joint EEG-MEG (EMEG) training yields consistent improvements across both modalities. Code and checkpoints are publicly available at https://github.com/OpenTSLab/BrainOmni.

Figures (6)

• Figure 1: Illustration of the training pipeline of BrainTokenizer. Left: Overview of the autoencoder scheme for BrainTokenizer training. Middle: Structure of the BrainTokenizer and Reconstructor. Right: Structure of the Sensor Encoder.
• Figure 2: Illustration of the training framework of BrainOmni.
• Figure 3: (a) Trend of L1 loss over the number of latent source variables. (b) The training loss and validation loss curves during the training phase of BrainTokenizer. (c) The accuracy curves for parallel mask prediction on the labels of each codebook layer during the training phase of BrainOmni on the training set.
• Figure 4: The waveforms and topographies of the reconstructed and original EEG signals: (a) the standard 10-20 system that was seen during the pre-training phase; (b) the Synnaps system not seen during the pretraining phase; (c) the NeuroImaging system not seen during the pretraining phase.
• Figure 5: Correlation between source current estimation and the proposed BrainTokenizer. The BrainTokenizer (excluding RVQ) can be viewed as the backward solution in source current estimation, and the reconstructor can be viewed as the forward solution.