Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Advancing Brainwave Modeling with a Codebook-Based Foundation Model

Konstantinos Barmpas, Na Lee, Yannis Panagakis, Dimitrios A. Adamos, Nikolaos Laskaris, Stefanos Zafeiriou

TL;DR

LaBraM++ addresses the core limitation of prior EEG foundation models in capturing neural oscillations by enforcing a phase representation that respects circular topology. It replaces the direct phase loss with sine/cosine phase losses and a phase-representing vector $r(\phi)=(\sin\phi,\cos\phi)$, improving gradient stability and information retention. The approach also adds Common Average Reference, per-patch Z-scoring, and refined temporal/spatial embeddings, yielding substantial gains over LaBraM in pre-training efficiency and downstream EEG classification across multiple datasets, and achieving competitive results with open-source LBMs. This work demonstrates how principled signal-processing foundations can be integrated into large brainwave models to enhance generalization and performance in diverse BCI tasks.

Abstract

Recent advances in large-scale pre-trained Electroencephalogram (EEG) models have shown great promise, driving progress in Brain-Computer Interfaces (BCIs) and healthcare applications. However, despite their success, many existing pre-trained models have struggled to fully capture the rich information content of neural oscillations, a limitation that fundamentally constrains their performance and generalizability across diverse BCI tasks. This limitation is frequently rooted in suboptimal architectural design choices which constrain their representational capacity. In this work, we introduce LaBraM++, an enhanced Large Brainwave Foundation Model (LBM) that incorporates principled improvements grounded in robust signal processing foundations. LaBraM++ demonstrates substantial gains across a variety of tasks, consistently outperforming its originally-based architecture and achieving competitive results when compared to other open-source LBMs. Its superior performance and training efficiency highlight its potential as a strong foundation for future advancements in LBMs.

Advancing Brainwave Modeling with a Codebook-Based Foundation Model

TL;DR

LaBraM++ addresses the core limitation of prior EEG foundation models in capturing neural oscillations by enforcing a phase representation that respects circular topology. It replaces the direct phase loss with sine/cosine phase losses and a phase-representing vector , improving gradient stability and information retention. The approach also adds Common Average Reference, per-patch Z-scoring, and refined temporal/spatial embeddings, yielding substantial gains over LaBraM in pre-training efficiency and downstream EEG classification across multiple datasets, and achieving competitive results with open-source LBMs. This work demonstrates how principled signal-processing foundations can be integrated into large brainwave models to enhance generalization and performance in diverse BCI tasks.

Abstract

Recent advances in large-scale pre-trained Electroencephalogram (EEG) models have shown great promise, driving progress in Brain-Computer Interfaces (BCIs) and healthcare applications. However, despite their success, many existing pre-trained models have struggled to fully capture the rich information content of neural oscillations, a limitation that fundamentally constrains their performance and generalizability across diverse BCI tasks. This limitation is frequently rooted in suboptimal architectural design choices which constrain their representational capacity. In this work, we introduce LaBraM++, an enhanced Large Brainwave Foundation Model (LBM) that incorporates principled improvements grounded in robust signal processing foundations. LaBraM++ demonstrates substantial gains across a variety of tasks, consistently outperforming its originally-based architecture and achieving competitive results when compared to other open-source LBMs. Its superior performance and training efficiency highlight its potential as a strong foundation for future advancements in LBMs.

Paper Structure

This paper contains 20 sections, 10 equations, 3 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Model
  4. Description
  5. Theoretical Basis
  6. Input Patches
  7. Experiments
  8. Implementation Details
  9. Data Preprocessing
  10. Temporal and Spatial Embeddings
  11. Hyperparameters
  12. Comparison with LaBraM
  13. Signal Reconstruction
  14. Foundation Model Training Loss
  15. Fine-Tuning Performance
  16. ...and 5 more sections

Figures (3)

  • Figure 1: Illustration of LaBraM++ modified tokenizer. This tokenizer discretizes EEG signals into neural tokens by reconstructing the Fourier amplitude, the sine and the cosine of the Fourier phase.
  • Figure 2: Reconstructed EEG signals from LaBraM (red) and LaBraM++ (green) tokenizers. Results show improved reconstructions after applying our modifications. Blue lines denote the input EEG signal.
  • Figure 3: The pre-training loss curve for LaBraM and LaBraM++