CwA-T: A Channelwise AutoEncoder with Transformer for EEG Abnormality Detection

TL;DR

This work tackles the challenge of detecting EEG abnormalities from long-term, high-dimensional recordings by introducing CwA-T, a hybrid framework that combines a channelwise CNN-based autoencoder with a lightweight single-head transformer classifier. The channelwise autoencoder preserves channel independence while drastically reducing data dimensionality, enabling efficient compression, and the single-head transformer models long-range dependencies in the latent space. On the TUH Abnormal EEG Corpus, CwA-T achieves strong per-case accuracy ($85.0\%$) with high specificity ($91.2\%$) and balanced sensitivity ($76.2\%$), while requiring only $202$M FLOPs and $2.9$M parameters, outperforming several baselines and substantially reducing computational cost compared with a full transformer. The framework maintains interpretability through its channelwise structure, and results suggest potential for real-time clinical deployment and advanced neuroscience research, with future work exploring attention-based connectivity and multi-modal extensions.

Abstract

Electroencephalogram (EEG) signals are critical for detecting abnormal brain activity, but their high dimensionality and complexity pose significant challenges for effective analysis. In this paper, we propose CwA-T, a novel framework that combines a channelwise CNN-based autoencoder with a single-head transformer classifier for efficient EEG abnormality detection. The channelwise autoencoder compresses raw EEG signals while preserving channel independence, reducing computational costs and retaining biologically meaningful features. The compressed representations are then fed into the transformer-based classifier, which efficiently models long-term dependencies to distinguish between normal and abnormal signals. Evaluated on the TUH Abnormal EEG Corpus, the proposed model achieves 85.0% accuracy, 76.2% sensitivity, and 91.2% specificity at the per-case level, outperforming baseline models such as EEGNet, Deep4Conv, and FusionCNN. Furthermore, CwA-T requires only 202M FLOPs and 2.9M parameters, making it significantly more efficient than transformer-based alternatives. The framework retains interpretability through its channelwise design, demonstrating great potential for future applications in neuroscience research and clinical practice. The source code is available at https://github.com/YossiZhao/CAE-T.

Figures (5)

• Figure 1: Overview of the proposed CwA-T framework for EEG-based abnormality detection. The framework includes three primary stages: (i) Preprocessing, where raw EEG signals are downsampled, segmented using a sliding window, and z-normalized to reduce noise and standardize inputs; (ii) Channelwise Autoencoder, which compresses the preprocessed signals into latent representations through independent channelwise feature extraction, preserving the natural structure of EEG data across multiple layers; and (iii) Single-Head Transformer Classifier, which utilizes scaled dot-product attention to identify patterns in the compressed representations and outputs a classification of the signals as normal or pathological. The output also has potential to support interpretability via connectivity analysis and visualization.
• Figure 2: Overview of the channelwise autoencoder architecture. The autoencoder processes input feature maps independently for each EEG channel to preserve channelwise independence. Each input feature map $F$ from the $k-1th$ layer is passed through a 1D convolutional layer with grouped convolutions ( $\text{groups} = C$ , where $C$ is the number of channels), followed by an addition operation and Layer Normalization ($Add\&LayerNorm$). This ensures computational efficiency and prevents inter-channel dependencies. The downsampling operation further reduces the spatial dimension of the feature maps, producing the output feature map $G$ for the $k -th$ layer. This design aligns with the natural structure of EEG data for neuroscience research and clinical applications.
• Figure 3: The electrode placement in the standard 10-20 system. In our study, we use the signals from Fp1, Fp2, F3, F4, F7, F8, C3, C4, T7, T8, P3, P4, P7, P8, O1, O2, Fz, Cz, and Pz.
• Figure 4: Performance comparison of classifiers between MLPs and Single-head transformer
• Figure 5: Examples of EEG signals with annotated abnormalities and corresponding autoencoder outputs. This figure illustrates samples of EEG signals from abnormal patients alongside the outputs of the channelwise autoencoder. The raw EEG signal (blue line), autoencoder output (red line), and corresponding spectrograms are shown for 2-minute segments. Anomalous points, identified by rectangles and circles, suggest potential abnormalities detected by the model.