Spatial-Functional awareness Transformer-based graph archetype contrastive learning for Decoding Visual Neural Representations from EEG

TL;DR

Decoding visual neural representations from EEG is hindered by high-dimensional, noisy, non-Euclidean signals and substantial inter-subject variability. The paper introduces Spatial-Functional Awareness Transformer-based Graph Archetype Learning (SFTG), combining an EEG Graph Transformer (EGT) that fuses spatial electrode connectivity with temporal dynamics and Graph Archetype Contrastive Learning (GAC) to learn subject-specific archetypes. The EGT employs Laplacian-based positional encodings and full-relation attention to capture complex spatiotemporal dependencies, while GAC enforces alignment between sequence- and channel-level EEG representations and subject archetypes via a dual-level contrastive loss. Evaluations on the Things-EEG dataset show superior performance in both subject-dependent and subject-independent settings, supported by qualitative analyses (RSA, t-SNE) that reveal proper clustering and alignment with visual concepts. Overall, the work demonstrates that integrating graph-structured EEG modeling with contrastive objectives yields more robust and generalizable brain representations for EEG-based visual decoding, with meaningful implications for robust brain–computer interfaces.

Abstract

Decoding visual neural representations from Electroencephalography (EEG) signals remains a formidable challenge due to their high-dimensional, noisy, and non-Euclidean nature. In this work, we propose a Spatial-Functional Awareness Transformer-based Graph Archetype Contrastive Learning (SFTG) framework to enhance EEG-based visual decoding. Specifically, we introduce the EEG Graph Transformer (EGT), a novel graph-based neural architecture that simultaneously encodes spatial brain connectivity and temporal neural dynamics. To mitigate high intra-subject variability, we propose Graph Archetype Contrastive Learning (GAC), which learns subject-specific EEG graph archetypes to improve feature consistency and class separability. Furthermore, we conduct comprehensive subject-dependent and subject-independent evaluations on the Things-EEG dataset, demonstrating that our approach significantly outperforms prior state-of-the-art EEG decoding methods.The results underscore the transformative potential of integrating graph-based learning with contrastive objectives to enhance EEG-based brain decoding, paving the way for more generalizable and robust neural representations.

Figures (4)

• Figure 1: Spatial and functional connectivity across 63 EEG channels is quantified by calculating the Pearson correlation coefficient.
• Figure 2: Our framework consists of the EEG Graph Transformer (EGT) (blue box) and Graph Archetype Contrastive Learning (GAC) (orange box). We construct EEG graphs by leveraging spatial and functional connectivity, transforming raw EEG signals into node and graph embeddings with additional positional encodings. the outputs of the FR heads are integrated into transformer encoder to enhance feature extraction. The encoded EEG features are projected and trained to align with visual representations.
• Figure 3: Semantic Similarity Analysis and Visualization. (A) shows the cosine similarity computed for feature pairs of 200 test-set concepts. (B) displays the classification outcomes, with the ground truth presented in the first column and the top-5 predictions in the subsequent columns.
• Figure 4: t-SNE visualization of five categories: animal, food, vehicle, tool, and others. (A) EEG feature distribution of SFTG training results. (B) Corresponding images in the t-SNE space.