MEG-GPT: A transformer-based foundation model for magnetoencephalography data

TL;DR

MEG-GPT presents a foundation-model approach to magnetoencephalography by coupling a data-driven, lossless tokeniser with a transformer-based autoregressive model trained on large-scale resting-state MEG data. The tokeniser enables discrete, high-resolution representation of continuous MEG signals, while the MEG-GPT model learns complex spatiotemporal dependencies and subject-specific patterns through time-attention and next-time-point prediction. The framework demonstrates realistic synthetic MEG generation, captures bursting dynamics and inter-subject variability, and delivers substantial gains in zero-shot decoding across sessions and subjects; fine-tuning further improves performance on new datasets. Together, these results establish MEG-GPT as a scalable foundation model for electrophysiological data with broad implications for neural decoding and computational neuroscience.

Abstract

Modelling the complex spatiotemporal patterns of large-scale brain dynamics is crucial for neuroscience, but traditional methods fail to capture the rich structure in modalities such as magnetoencephalography (MEG). Recent advances in deep learning have enabled significant progress in other domains, such as language and vision, by using foundation models at scale. Here, we introduce MEG-GPT, a transformer based foundation model that uses time-attention and next time-point prediction. To facilitate this, we also introduce a novel data-driven tokeniser for continuous MEG data, which preserves the high temporal resolution of continuous MEG signals without lossy transformations. We trained MEG-GPT on tokenised brain region time-courses extracted from a large-scale MEG dataset (N=612, eyes-closed rest, Cam-CAN data), and show that the learnt model can generate data with realistic spatio-spectral properties, including transient events and population variability. Critically, it performs well in downstream decoding tasks, improving downstream supervised prediction task, showing improved zero-shot generalisation across sessions (improving accuracy from 0.54 to 0.59) and subjects (improving accuracy from 0.41 to 0.49) compared to a baseline methods. Furthermore, we show the model can be efficiently fine-tuned on a smaller labelled dataset to boost performance in cross-subject decoding scenarios. This work establishes a powerful foundation model for electrophysiological data, paving the way for applications in computational neuroscience and neural decoding.

Figures (16)

• Figure 1: Illustration of the tokeniser. The input to the tokeniser is a single sequence of continuous MEG data and its encoder's output is a single sequence of token labels. The tokeniser is based on an autoencoder framework. The encoder maps the continuous data $\boldsymbol{x}$ onto logits $\boldsymbol{\alpha}$ that are used to calculate token labels $\boldsymbol{z}$. The decoder reconstructs the data $\tilde{\boldsymbol{x}}$ from token labels $\boldsymbol{z}$ using a weighted sum of token kernels $\boldsymbol{e}$. The token labels $\boldsymbol{z}$ are then fed into MEG-GPT.
• Figure 2: MEG-GPT foundation model. MEG-GPT is a nonlinear autoregressive model that predicts the token at the next timepoint from a preceding sequence of token labels. However, unlike classical autoregressive models, the autoregressive weighting can change as a function of the data. A key part of the foundation model is learning an embedding space for the tokens, which generates vector for each token $\boldsymbol{v}_z$, channel $\boldsymbol{v}_c$, temporal position $\boldsymbol{v}_p$, any additional information $\boldsymbol{v}_s$ (e.g. subject ID) and combines (adds) these to provide a single embedding vector $\boldsymbol{v}$ at each time point. This combined embedding vector is fed into the Transformer Decoder, and the output of the Transformer Decoder is used to predict the next token via the Prediction Head.
• Figure 3: Transformer Decoder and the masked multi-head attention used in MEG-GPT. A) Architecture of the Transformer Decoder in Figure \ref{['fig:meg-gpt']}. B) Masked multi-head (self-)attention used in the Transformer Decoder. Shows an example of predicting the token at the next time step with a receptive field of $L = 12$. The input embeddings $\boldsymbol{v}$ are divided into 3 patches of patch size $L_p = 4$, resulting in patched inputs$\boldsymbol{p}^*$. The last $L_u = 3$ unpatched embeddings are retained as the unpatched inputs$\boldsymbol{v}^*$, allowing for finer temporal resolution near the prediction point. The attention matrix used in the self-attention mechanism has a latent sequence length of $L_\mathrm{latent} = 5$ corresponding to the output tokens. Each output attends to both patched inputs $\boldsymbol{p}^*$ and unpatched inputs $\boldsymbol{v}^*$. A temporal mask (indicated by black squares) is applied to prevent information leakage from future time points during training.
• Figure 4: The data-adaptive tokeniser reconstructs parcellated MEG data with high accuracy and generalises well to unseen data. A) Original signals (black) and tokeniser reconstructions (where the original signals are tokenised and then de-tokenised) (red) are shown for each session from the Cam-CAN training set (top row), Cam-CAN testing set (middle row), and the Wakeman-Henson dataset (bottom row). Only the first 1.2 s of each session are displayed. B) Percentage of variance explained (PVE) of the reconstructed data across different datasets.
• Figure 5: MEG-GPT captures spatial and spectral characteristics of real data. Each plot is calculated for the real data, MEG-GPT generated data, and linear AR model generated data. A) Group-average PSD for each parcel. The glass brain plot in the top right indicates the location of each parcel. B) Narrow-band power maps relative to the average across frequency bands.