EEG-to-fMRI synthesis of task-evoked and spontaneous brain activity: addressing issues of statistical significance and generalizability

TL;DR

This study tackles the challenge of EEG-to-fMRI synthesis by evaluating whether EEG features can predict task-evoked and spontaneous BOLD activity in the somatomotor network with statistical significance and across days. It introduces subject-specific distributed-lag linear models with Sparse Group LASSO, leverages a two-session design to test generalizability, and uses rigorous surrogate-based significance testing to guard against data leakage. Results show robust significance for task-evoked activity and more modest but present predictions for spontaneous fluctuations, with clear evidence of leakage when train-test data come from the same session. The work advances interpretability and methodological rigor in EEG-to-fMRI synthesis and highlights practical considerations for future neurofeedback and cross-session generalization.

Abstract

A growing interest has developed in the problem of training models of EEG features to predict brain activity measured using fMRI, i.e. the problem of EEG-to-fMRI synthesis. Despite some reported success, the statistical significance and generalizability of EEG-to-fMRI predictions remains to be fully demonstrated. Here, we investigate the predictive power of EEG for both task-evoked and spontaneous activity of the somatomotor network measured by fMRI, based on data collected from healthy subjects in two different sessions. We trained subject-specific distributed-lag linear models of time-varying, multi-channel EEG spectral power using Sparse Group LASSO regularization, and we showed that learned models outperformed conventional EEG somatomotor rhythm predictors as well as massive univariate correlation models. Furthermore, we showed that learned models were statistically significantly better than appropriate null models in most subjects and conditions, although less frequently for spontaneous compared to task-evoked activity. Critically, predictions improved significantly when training and testing on data acquired in the same session relative to across sessions, highlighting the importance of temporally separating the collection of train and test data to avoid data leakage and optimistic bias in model generalization. In sum, while we demonstrate that EEG models can provide fMRI predictions with statistical significance, we also show that predictive power is impaired for spontaneous fluctuations in brain activity and for models trained on data acquired in a different session. Our findings highlight the need to explicitly consider these often overlooked issues in the growing literature of EEG-to-fMRI synthesis.

Figures (13)

• Figure 1: Experimental protocol. (A) Illustration of the NeuRow, Graz and Rest scenarios, as presented to the participant in the screen. (B) Structure of one run (top) and one task block (bottom), for the NeuRow scenario. Blocks of left and right arm MI are separated by blocks of rest. Each trial is initiated with a preparation period and ends with a brief rest before the next trial. For the Graz scenario, the protocol timings are identical but only the arrow is presented to the subject as visual stimulus.
• Figure 2: Feature and Signal Extraction Pipelines. (A) Relative power timeseries are extracted from EEG for each channel and frequency bin. From here task data is treated as the TE response, from which we subtract the average trial response centered on the trial stimuli times to produce TBT fluctuations. Finally, TE, TBT, and Rest timeseries are resampled to match the fMRI TR, before being shifted and stored in a matrix to serve as lagged regressors in our downstream models. (B) Task fMRI data is decomposed into TE responses and TBT fluctuations by first-level GLM analysis, and then masked by a group task-activation map. The rest data is submitted to a group-level ICA decomposition, and SMN IC selection. Finally, the first level of dual regression extracts a subject-specific SMN BOLD timecourse which is analogous to those found in the case of task data.
• Figure 3: Nested Cross-Validation Schemes. (A) Two train-test splitting schemes were used: inter-session (keeping the two sessions separate) and intra-session (mixing the two sessions). (B) For hyperparameter optimization, a block cross-validation scheme was used, by splitting the training dataset into 3 folds. A model is then trained using the entire training dataset and the selected hyperparameters and then applied to the held out test dataset and the Pearsomn correlation between the model prediction and the true BOLD signal is computed to assess model performance.
• Figure 4: Group fMRI SMN maps. SMN maps identified for both tasks as the average motor imagery activation across subjects and sessions (Graz, Neurow), and for rest by group ICA and subsequent identification of the SMN network by template matching (Rest). The Z statistic maps obtained after thresholding for statistical significance (color) are overlaid on the MNI anatomical brain template (gray).
• Figure 5: fMRI results for a representative subject (A) fMRI SMN maps obtained for each task (Graz and NeuRow) and for resting state (Rest). (B) BOLD-fMRI time series (black) and respective model predictions (red), for both task-evoked (TE) and trial-by-trial (TBT) activities for the tasks (Graz - TE, Graz - TBT, Neurow - TE, NeuRow - TBT), with trial onset times indicated by the dashed lines (orange -- left, blue -- right).