Self-Calibrating BCIs: Ranking and Recovery of Mental Targets Without Labels

TL;DR

The paper introduces CURSOR, a fully self-calibrating BCI framework that recovers a participant's mental target in a continuous latent space using unlabeled EEG–image pair data. By defining a distance-based similarity and a relative RMSE-based scoring function, CURSOR ranks and optimizes hypothetical targets without supervision, enabling target recovery and even generation of indistinguishable stimuli. Empirical results on a large EEG-face dataset show strong alignment with human perceptual judgments and ground-truth distances, with ground-truth label recovery approaching supervised performance. The work also provides a substantial EEG-face dataset and demonstrates the method's robustness across estimators and data regimes, highlighting the potential and ethical considerations of label-free SC-BCIs in continuous domains.

Abstract

We consider the problem of recovering a mental target (e.g., an image of a face) that a participant has in mind from paired EEG (i.e., brain responses) and image (i.e., perceived faces) data collected during interactive sessions without access to labeled information. The problem has been previously explored with labeled data but not via self-calibration, where labeled data is unavailable. Here, we present the first framework and an algorithm, CURSOR, that learns to recover unknown mental targets without access to labeled data or pre-trained decoders. Our experiments on naturalistic images of faces demonstrate that CURSOR can (1) predict image similarity scores that correlate with human perceptual judgments without any label information, (2) use these scores to rank stimuli against an unknown mental target, and (3) generate new stimuli indistinguishable from the unknown mental target (validated via a user study, N=53).

Figures (13)

• Figure 1: (Left) Participants are instructed to hold a mental target $x^*$, represented as $z^*$, unknown to our system, while a sequence of $i = \{1, \ldots, N\}$ images are shown to them and their EEG responses are being recorded. Cursor's goal is to recover $z^*$ with this information alone and fully unsupervised. (Middle) Cursor generates hypothetical target images, e.g. $h_1$, and builds a dedicated estimation problem to predict the distance $d_i$ between $h_1$ and $z_i$ from the corresponding EEG response $e_i$, for all $i$. (Right) The performance of an estimator on this task is the score $S(h_1)$ attached to $h_1$. The hypothesis with the highest score $\hat{h} = \mathop{\mathrm{arg\,max}}\limits_{h \in \mathcal{H}} S(h)$ is assumed to be the unknown mental target. Images are generated from $z_i \in \mathbb{R}^{512}$ embeddings with EEG responses $e_i \in \mathbb{R}^{203}$. We collected $N=9234$$\{z_i, e_i\}$ pairs for this study from 29 participants.
• Figure 2: (Left) Cursor and human scores correlate strongly with similarity. [Blue] Predicted scores ($S$) (Alg. \ref{['alg:cursor_scoring']} with LR estimator using full dataset) against ground-truth similarity ($d$) for 60 face images with regression lines and 95% CI. [Pink] Human-assigned H-Rank scores to sets of 10 facial images morphing away from the target [right axis]. (Right) A face at $d\leq1.6$ has an error rate at chance level, suggesting perceptual indistinguishability to the target. A stimulus at $d\geq16.3$ has an error rate nearing 0, confirming perceivable differences from the target. These statistics (Wilson Score shading at 95% CI) were collected during H-ID trials [top]. The vertical dotted lines show ranking and optimization performance, both recovering faces indistinguishable from the target.
• Figure 3: Rank-related performance for different estimators and data sizes (mean $\pm$ ste, std at $N$, inv. y-axis). (Left) Correlation coefficient ($R$) between scores and ground-truth distances. (Middle) Target rank out of 60. (Right) Euclidean distance to the target for top ranked candidate. The bottom dotted line is theoretical random performance. With LR rankings, human subjects could barely distinguish top-ranked faces from the target face with $\eta(3.6) = 0.377 \, {\text{(95% CI: 0.304, 0.455)}}$.
• Figure 4: (Left) Optimization converges to $<1$ distance to the target. Candidates distance ($d$) to the target against iteration number (left, mean $\pm$ std). We can recover near-target faces even when near-target stimuli are absent in the dataset, up to $d_{removed} =10$. Best candidates found after 1000 iterations shown against ablation distances $d_{removed}$ around the target (right, mean $\pm$ std). (Right) Top-ranked (middle) and optimized (right) images per estimators from one randomly selected run and four randomly selected target faces (left). Images from Cursor using Linear Regression (LR) are indistinguishable from the target image, while controls S-LR and Dummy exhibit visual differences.
• Figure 5: PCA component sweeps for (Left) EEG signals and (Right) face embeddings.