NeuroRVQ: Multi-Scale EEG Tokenization for Generative Large Brainwave Models

TL;DR

NeuroRVQ tackles the critical tokenizer bottleneck in EEG-based large brainwave models by introducing a codebook-based tokenizer that performs multi-scale temporal encoding and per-scale residual vector quantization, coupled with a phase- and amplitude-aware training loss. This design yields faithful reconstruction across all frequency bands and enables scalable pretraining of a foundation-style backbone on tokenized EEG. Empirical results demonstrate state-of-the-art reconstruction quality, robust out-of-distribution generalization, and superior downstream classification accuracy with a compact backbone compared to larger precedents. The work suggests the codebook-based tokenizer can generalize to other biosignals, advancing practical biosignal decoding and multimodal integration.

Abstract

Electroencephalography (EEG) captures neural activity across multiple temporal and spectral scales, yielding signals that are rich but complex for representation learning. Recently, EEG foundation models trained to predict masked signal-tokens have shown promise for learning generalizable representations. However, their performance is hindered by their signal tokenization modules. Existing neural tokenizers fail to preserve high-frequency dynamics, limiting their ability to reconstruct EEG signals with high fidelity. We introduce NeuroRVQ, a scalable Large Brainwave Model (LBM) centered on a codebook-based tokenizer. Our tokenizer integrates: (i) multi-scale feature extraction modules that capture the full frequency neural spectrum; (ii) hierarchical residual vector quantization (RVQ) codebooks for high-resolution encoding; and, (iii) an EEG signal phase- and amplitude-aware loss function for efficient training. This design enables efficient EEG compression while supporting accurate reconstruction across all frequency bands, leading to robust generative masked modeling. Our empirical results demonstrate that NeuroRVQ achieves lower reconstruction error and outperforms existing LBMs on a variety of downstream tasks. More broadly, NeuroRVQ tokenizer establishes a strong prior for codebook-based general-purpose brainwave models, enabling advances in neural decoding, generative modeling and multimodal biosignal integration.

Figures (8)

• Figure 1: a) The NeuroRVQ tokenizer model architecture; b) the multi-scale temporal encoder architecture; and c) the temporal branch architecture. The multi-scale temporal encoder extracts multi-scale temporal features that are passed to the transformer encoder. RVQ codebooks per temporal branch discretize the EEG patches into neural tokens. The tokenizer decoder reconstructs the EEG signal by using the Fourier spectrum.
• Figure 2: Comparison of phase modeling approaches: (a) Independent MSE can produce invalid predictions (b) Unit-circle-aware loss enforces valid phase angles.
• Figure 3: NeuroRVQ Foundation Model. During pre-training, random EEG patches are masked, and the model is trained to reconstruct the missing tokens from the RVQ codebook using the surrounding visible patches as context.
• Figure 4: Per-Band Analysis of Reconstructed EEG signal from LaBraM and NeuroRVQ codebook-based tokenizers. Green lines denote the input EEG signal, orange the reconstructed EEG signal for LaBraM (ours) and red the reconstructed EEG signal for NeuroRVQ.
• Figure 5: Validation MSE reconstruction loss for the NeuroRVQ tokenizer with different numbers of codebooks (layers) per RVQ.