A Unified SPD Token Transformer Framework for EEG Classification: Systematic Comparison of Geometric Embeddings

TL;DR

A unified Transformer framework comparing BWSPD, Log-Euclidean, and Euclidean embeddings within identical architecture across 1,500+ runs on three EEG paradigms achieves state-of-the-art performance on all datasets, substantially outperforming classical Riemannian classifiers and recent SPD baselines.

Abstract

Spatial covariance matrices of EEG signals are Symmetric Positive Definite (SPD) and lie on a Riemannian manifold, yet the theoretical connection between embedding geometry and optimization dynamics remains unexplored. We provide a formal analysis linking embedding choice to gradient conditioning and numerical stability for SPD manifolds, establishing three theoretical results: (1) BWSPD's $\sqrtκ$ gradient conditioning (vs $κ$ for Log-Euclidean) via Daleckii-Kreĭn matrices provides better gradient conditioning on high-dimensional inputs ($d \geq 22$), with this advantage reducing on low-dimensional inputs ($d \leq 8$) where eigendecomposition overhead dominates; (2) Embedding-Space Batch Normalization (BN-Embed) approximates Riemannian normalization up to $O(\varepsilon^2)$ error, yielding $+26\%$ accuracy on 56-channel ERP data but negligible effect on 8-channel SSVEP data, matching the channel-count-dependent prediction; (3) bi-Lipschitz bounds prove BWSPD tokens preserve manifold distances with distortion governed solely by the condition ratio $κ$. We validate these predictions via a unified Transformer framework comparing BWSPD, Log-Euclidean, and Euclidean embeddings within identical architecture across 1,500+ runs on three EEG paradigms (motor imagery, ERP, SSVEP; 36 subjects). Our Log-Euclidean Transformer achieves state-of-the-art performance on all datasets, substantially outperforming classical Riemannian classifiers and recent SPD baselines, while BWSPD offers competitive accuracy with similar training time.

Figures (5)

• Figure 1: Unified SPD Token Transformer. Only the embedding layer (blue/orange/green) differs; projection, BN-Embed, Transformer encoder, and classifier are shared.
• Figure 2: (a) Gradient conditioning theory: BWSPD has $\sqrt{\kappa}$ conditioning vs $\kappa$ for Log-Euclidean, providing better gradient conditioning on high-dimensional inputs ($d \geq 22$ channels). (b) Convergence speed comparison on BCI2a: Both embeddings achieve similar training times, with Log-Euclidean achieving higher final accuracy.
• Figure 3: Embedding space visualization (t-SNE) for BCI2a Subject 1. Geometric embeddings (BWSPD, Log-Euclidean) show better class separation than Euclidean, validating that geometric structure is preserved in the token space.
• Figure 4: Training curves (train/val/test accuracy vs epoch) for all three embedding methods across datasets. Log-Euclidean shows slower convergence but higher final accuracy; BWSPD converges faster with competitive performance.
• Figure 5: Confusion matrices for representative BCI2a subjects (50 epochs, bandpass 4--40 Hz, seed=42 for consistency). Left: Subject 2 (seed=42: 98.81%; mean across 5 seeds: 99.76%$\pm$0.22%) represents high-performing subjects with consistent performance across seeds. Right: Subject 4 (seed=42: 98.81%; mean: 87.70%$\pm$27.51%, range: 38.49%--100.00%) represents subjects with high variance. Note: While seed=42 for S4 achieves near-maximum accuracy (98.81%), this is not representative of the mean (87.70%); the mean accuracy across all seeds is the more reliable metric. The high variance (std: 27.51%) reflects the difficulty of motor imagery classification for this subject, with some seeds showing confusion between left and right hand classes.

Theorems & Definitions (29)

• Theorem 3.1: Bi-Lipschitz Embedding — informal; see \ref{['thm:distortion_commuting', 'thm:distortion_general']}
• Theorem 3.2: Gradient Conditioning — informal; see \ref{['thm:K_conditioning']}
• Proposition 3.3: BN-Embed $\approx$ Riemannian Normalization — informal; see \ref{['prop:barycenter_approx']}
• Definition 12.1: Bures-Wasserstein Distance
• Lemma 12.2: Norm Equivalence for Symmetric Matrices
• proof
• Theorem 12.3: Distortion Bounds: Commuting Case
• proof
• Theorem 12.4: Distortion Bounds: General Case
• proof