LUNA: Efficient and Topology-Agnostic Foundation Model for EEG Signal Analysis

TL;DR

LUNA addresses the critical challenge of topological heterogeneity in EEG by learning a topology-agnostic encoder that maps variable electrode layouts into a fixed latent space via learned queries and cross-attention, enabling downstream temporal Transformers to operate efficiently on a compact representation. Pre-trained on over 21,000 hours of multi-montage data, LUNA demonstrates strong performance across abnormality, artifact, slowing, and emotion recognition benchmarks, while achieving substantial efficiency gains (linear-in-channel unification and patch-wise temporal attention) that reduce FLOPs and memory usage. The work provides extensive ablation and latent-space analyses, confirming the utility of learned spatial queries and frequency features, though it notes some limitations in zero-shot transfer to unseen high-density topologies. Overall, LUNA showcases the potential of topology-agnostic latent representations for scalable EEG modeling, with implications for transfer learning across datasets and real-time, resource-constrained deployment.

Abstract

Electroencephalography (EEG) offers a non-invasive lens into human brain activity, but building large-scale models is hampered by topological heterogeneity: each public EEG data defines its own electrode layout, limiting generalization. We introduce LUNA (Latent Unified Network Architecture), a self-supervised foundation model that reconciles disparate electrode geometries while scaling linearly -- not quadratically -- with channel count. LUNA compresses multi-channel EEG into a fixed-size, topology-agnostic latent space via learned queries and cross-attention. Downstream transformer blocks then operate exclusively on this latent representation using patch-wise temporal self-attention, decoupling computation from electrode count. Pre-trained on TUEG and Siena (over 21,000 hours of raw EEG across diverse montages) using a masked-patch reconstruction objective, LUNA transfers effectively to four downstream tasks: abnormality detection, artifact rejection, slowing classification, and emotion recognition. It demonstrates highly competitive performance across several benchmarks, achieving state-of-the-art results on TUAR and TUSL, e.g., 0.921 AUROC on TUAR, while reducing FLOPs by 300x and trimming GPU memory use by up to 10x. Critically, these gains are consistent across all evaluated electrode configurations. Code is available at https://github.com/pulp-bio/BioFoundation

Figures (10)

• Figure 1: Overview of LUNA. EEG signals ($B \times C \times T$) are segmented into patches and embedded. Channel-Unification Module maps channel-wise features into a fixed-size latent space using learned queries ($Q$). Patch-wise Temporal Attention processes this latent sequence. The decoder generates task-specific outputs.
• Figure 2: Computational cost scaling of LUNA and baseline models. (a) FLOPs and Memory usage vs. number of patches. (b) FLOPs and Memory usage vs. number of channels. LUNA demonstrates significantly better efficiency and scalability, especially compared to full attention (LaBraM), and favorable scaling compared to alternating attention (CBraMod) due to the fixed latent query space. CBraMod has a variable sized decoder based on the number of patches and channels; therefore, its model size as well as its resource usage grows rapidly. FLOPs are measured with fvcore's FlopCountAnalysis over 50 random inputs, including encoder+decoder, with window length $T$, patch size $P$, and reported as GFLOPs per forward pass.
• Figure 3: t-SNE of LUNA-Base embeddings on downstream datasets before fine-tuning.
• Figure 4: Visualization of the attention patterns of queries in LUNA-Base on Siena detti2020siena topology.
• Figure 5: Loss curves during pre-training for LUNA-Base (Reconstruction and Query Specialization Loss).