Confidence-Aware Neural Decoding of Overt Speech from EEG: Toward Robust Brain-Computer Interfaces

TL;DR

This work tackles the challenge of decoding overt speech from non-invasive EEG by introducing a confidence-aware pipeline that combines deep ensembles of compact EEG-specific CNNs, post-hoc temperature calibration, and an accept/abstain decision rule. It leverages leakage-safe block stratification to provide realistic operating points and evaluates multiple uncertainty scores (entropy, mutual information, and top-two margin) to drive selective classification, reporting improved AURC, calibration (ECE, NLL, Brier), and per-class acceptance across a 13-class task. Key contributions include a modular, deployment-oriented architecture, demonstrated gains over baselines, and a principled, tunable framework for reliability in real-time brain-computer interfaces. The approach has practical implications for robust, transparent BCI communication, enabling controlled trade-offs between throughput and accuracy in noisy EEG environments.

Abstract

Non-invasive brain-computer interfaces that decode spoken commands from electroencephalogram must be both accurate and trustworthy. We present a confidence-aware decoding framework that couples deep ensembles of compact, speech-oriented convolutional networks with post-hoc calibration and selective classification. Uncertainty is quantified using ensemble-based predictive entropy, top-two margin, and mutual information, and decisions are made with an abstain option governed by an accuracy-coverage operating point. The approach is evaluated on a multi-class overt speech dataset using a leakage-safe, block-stratified split that respects temporal contiguity. Compared with widely used baselines, the proposed method yields more reliable probability estimates, improved selective performance across operating points, and balanced per-class acceptance. These results suggest that confidence-aware neural decoding can provide robust, deployment-oriented behavior for real-world brain-computer interface communication systems.