Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Temporal Context and Architecture: A Benchmark for Naturalistic EEG Decoding

Mehmet Ergezer

TL;DR

This work benchmarks five architectures (CNN, LSTM, EEGXF, S4, S5) for naturalistic EEG decoding on the HBN movie-watching dataset across long temporal contexts (8–128 s). It reveals a practical trade-off: S5 delivers high accuracy with far fewer parameters and faster training, whereas EEGXF offers more robust uncertainty handling under frequency shifts and OOD inputs. Across generalization tests, S5 demonstrates stronger cross-subject performance but can be overconfident on unseen tasks, while EEGXF remains conservative, better calibrated for OOD scenarios. The findings guide practical choices between peak performance and robust uncertainty in real-world EEG decoding applications and motivate validation on additional benchmarks.

Abstract

We study how model architecture and temporal context interact in naturalistic EEG decoding. Using the HBN movie-watching dataset, we benchmark five architectures, CNN, LSTM, a stabilized Transformer (EEGXF), S4, and S5, on a 4-class task across segment lengths from 8s to 128s. Accuracy improves with longer context: at 64s, S5 reaches 98.7%+/-0.6 and CNN 98.3%+/-0.3, while S5 uses ~20x fewer parameters than CNN. To probe real-world robustness, we evaluate zero-shot cross-frequency shifts, cross-task OOD inputs, and leave-one-subject-out generalization. S5 achieves stronger cross-subject accuracy but makes over-confident errors on OOD tasks; EEGXF is more conservative and stable under frequency shifts, though less calibrated in-distribution. These results reveal a practical efficiency-robustness trade-off: S5 for parameter-efficient peak accuracy; EEGXF when robustness and conservative uncertainty are critical.

Temporal Context and Architecture: A Benchmark for Naturalistic EEG Decoding

TL;DR

This work benchmarks five architectures (CNN, LSTM, EEGXF, S4, S5) for naturalistic EEG decoding on the HBN movie-watching dataset across long temporal contexts (8–128 s). It reveals a practical trade-off: S5 delivers high accuracy with far fewer parameters and faster training, whereas EEGXF offers more robust uncertainty handling under frequency shifts and OOD inputs. Across generalization tests, S5 demonstrates stronger cross-subject performance but can be overconfident on unseen tasks, while EEGXF remains conservative, better calibrated for OOD scenarios. The findings guide practical choices between peak performance and robust uncertainty in real-world EEG decoding applications and motivate validation on additional benchmarks.

Abstract

We study how model architecture and temporal context interact in naturalistic EEG decoding. Using the HBN movie-watching dataset, we benchmark five architectures, CNN, LSTM, a stabilized Transformer (EEGXF), S4, and S5, on a 4-class task across segment lengths from 8s to 128s. Accuracy improves with longer context: at 64s, S5 reaches 98.7%+/-0.6 and CNN 98.3%+/-0.3, while S5 uses ~20x fewer parameters than CNN. To probe real-world robustness, we evaluate zero-shot cross-frequency shifts, cross-task OOD inputs, and leave-one-subject-out generalization. S5 achieves stronger cross-subject accuracy but makes over-confident errors on OOD tasks; EEGXF is more conservative and stable under frequency shifts, though less calibrated in-distribution. These results reveal a practical efficiency-robustness trade-off: S5 for parameter-efficient peak accuracy; EEGXF when robustness and conservative uncertainty are critical.
Paper Structure (11 sections, 1 equation, 3 figures, 6 tables)

This paper contains 11 sections, 1 equation, 3 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Dataset and Preprocessing
  3. Model Architectures
  4. Structured State-Space Model (S5)
  5. EEGXF: A Stabilized Transformer Baseline
  6. Experiments
  7. Results
  8. Cross-Frequency Robustness
  9. Leave-One-Subject-Out Generalization
  10. Zero-Shot Cross-Task Generalization
  11. Discussion and Conclusion

Figures (3)

  • Figure 1: Test accuracy versus segment length (8–128 s). Single-seed comparison of five architectures (CNN, EEGXF, LSTM, S4, S5). S5 and CNN demonstrate the strongest performance scaling with temporal context; x-axis log-scaled.
  • Figure 2: Performance–efficiency trade-off at 64 s. Single-seed comparison of five architectures; accuracy vs. parameters (log-scale); marker size $\propto$ training time (min). CNN is fastest (4.4 M parameters) while S5 (0.2 M parameters) attains state-of-the-art accuracy with $\sim$20$\times$ fewer parameters than CNN; EEGXF is comparable in training time but less accurate.
  • Figure 3: Zero-shot generalization performance at different sampling frequencies. S5 shows a large drop in accuracy when tested on downsampled EEG, while EEGXF remains robust. This highlights a speed-accuracy-robustness trade-off across architectures.