On the challenges of detecting MCI using EEG in the wild

TL;DR

The paper critically assesses the practicality of EEG-based MCI detection in real-world clinical settings by comparing a large, expert-labeled CAUEEG dataset with a smaller, general-practice GENEEG dataset. It shows that models trained on small data exhibit high variance and overconfidence, and that cross-domain shifts substantially degrade performance, revealing fundamental limits due to overlap between MCI and control feature distributions. The study finds CNNs on time-domain data to be the most effective among tested architectures, but performances remain modest in the presence of domain shift and data scarcity, underscoring the need for high-quality data collection and uncertainty-aware methods. Overall, the work argues for realistic goals, robust cross-domain validation, and uncertainty-aware modeling to advance non-invasive EEG-based MCI detection toward practical clinical use.

Abstract

Recent studies have shown promising results in the detection of Mild Cognitive Impairment (MCI) using easily accessible Electroencephalogram (EEG) data which would help administer early and effective treatment for dementia patients. However, the reliability and practicality of such systems remains unclear. In this work, we investigate the potential limitations and challenges in developing a robust MCI detection method using two contrasting datasets: 1) CAUEEG, collected and annotated by expert neurologists in controlled settings and 2) GENEEG, a new dataset collected and annotated in general practice clinics, a setting where routine MCI diagnoses are typically made. We find that training on small datasets, as is done by most previous works, tends to produce high variance models that make overconfident predictions, and are unreliable in practice. Additionally, distribution shifts between datasets make cross-domain generalization challenging. Finally, we show that MCI detection using EEG may suffer from fundamental limitations because of the overlapping nature of feature distributions with control groups. We call for more effort in high-quality data collection in actionable settings (like general practice clinics) to make progress towards this salient goal of non-invasive MCI detection.

Figures (8)

• Figure 1: An overview of the neural networks used in this study. The CNN and Transformer models are used to model time-domain EEG signals while an MLP is used to model frequency domain transformed data.
• Figure 2: Topographical map of the active EEG channels. T5 and T6 were not used in the study.
• Figure 3: Conversion of time-domain EEG signals to frequency domain spectral features by discretizing the power density spectrum. For bands of size 1 Hz from 4-30 Hz, we get 26 band power features per channel.
• Figure 4: In our ablation experiments, we found that models tend to perform better with smaller contig lengths, on time-domain data with standardization. The effect of class balancing was negligible. This is how we settled on data configuration (contig length, normalization and class balancing) before our final experiments.
• Figure 5: Models get better with less variance in performance as the dataset size increases (first two plots). Moreover, models also become more calibrated with better uncertainty estimates (last two plots).