Benchmarking ERP Analysis: Manual Features, Deep Learning, and Foundation Models

TL;DR

The paper benchmarks ERP analysis across two practical tasks (stimulus classification and disease detection) using 12 public ERP datasets, comparing manual feature extraction, supervised deep learning, and pre-trained EEG foundation models within a unified preprocessing/training pipeline. It also investigates Transformer patch embeddings to identify ERP-friendly embedding designs. Across 36 evaluations, deep learning generally outperforms manual features, while existing EEG foundation models do not consistently outperform models trained from scratch; EEGConformer shows the strongest overall performance, and uni-variate patch embedding often yields the best results for ERP-specific Transformers. The findings underscore the need for ERP-tailored pretraining and backbone architectures to fully exploit modern DL approaches in ERP analysis, and the provided framework offers guidance for method selection and model design in future ERP research.

Abstract

Event-related potential (ERP), a specialized paradigm of electroencephalographic (EEG), reflects neurological responses to external stimuli or events, generally associated with the brain's processing of specific cognitive tasks. ERP plays a critical role in cognitive analysis, the detection of neurological diseases, and the assessment of psychological states. Recent years have seen substantial advances in deep learning-based methods for spontaneous EEG and other non-time-locked task-related EEG signals. However, their effectiveness on ERP data remains underexplored, and many existing ERP studies still rely heavily on manually extracted features. In this paper, we conduct a comprehensive benchmark study that systematically compares traditional manual features (followed by a linear classifier), deep learning models, and pre-trained EEG foundation models for ERP analysis. We establish a unified data preprocessing and training pipeline and evaluate these approaches on two representative tasks, ERP stimulus classification and ERP-based brain disease detection, across 12 publicly available datasets. Furthermore, we investigate various patch-embedding strategies within advanced Transformer architectures to identify embedding designs that better suit ERP data. Our study provides a landmark framework to guide method selection and tailored model design for future ERP analysis. The code is available at https://github.com/DL4mHealth/ERP-Benchmark.

Figures (5)

• Figure 1: ERP vs Spontaneous EEG. ERP is inherently different from Spontaneous EEG, requiring specific analysis methods.
• Figure 2: Pipeline of ERP Analysis. Input raw EEG data are preprocessed with a unified pipeline to get ERP trials, including removal of non-EEG channels, notch and band-pass filtering, bad channel interpolation, average re-referencing, artifact removal, resampling, baseline correction, trial epoching, and Z-score normalization. The processed ERP trials are loaded and passed to various models for training and classification, including manual feature extraction, supervised deep learning trained from scratch, and foundation models with pre-trained weights.
• Figure 3: Transformer Embedding Comparison. Three commonly used EEG Transformer patch embedding methods.
• Figure 4: a) Average performance rank of 15 methods across all 12 datasets and 3 evaluation metrics. For example, the value 3.96 for EEGConformer indicates an average rank of 3.96 over 36 evaluations. b) Rank heatmap of 15 methods across all 12 datasets and 3 evaluation metrics. Lower ranks and a deeper blue color indicate better performance in both plots a and b.
• Figure 5: F1 Score comparison among multi-variate, uni-variate, and whole-variate token embedding commonly used in existing EEG Transformer. All other Transformer components are kept identical.