From Brainwaves to Brain Scans: A Robust Neural Network for EEG-to-fMRI Synthesis
Kristofer Grover Roos, Atsushi Fukuda, Quan Huu Cap
TL;DR
The paper addresses the challenge of generating fMRI-like images from EEG data by introducing E2fNet, a compact encoder–UNet–decoder CNN that translates multi-channel EEG spectro-temporal features into fMRI volumes. The architecture preserves electrode topology, extracts multi-scale EEG representations with a U‑Net, and reconstructs volumetric fMRI via a dedicated decoder, trained with a combined SSIM and MSE loss. Across three public EEG–fMRI datasets (NODDI, Oddball, CN-EPFL), E2fNet achieves state-of-the-art SSIM, outperforms CNN and transformer baselines, and shows more stable training than GAN-based approaches. This work suggests a cost-effective path for augmenting neuroimaging capabilities, with code available at https://github.com/kgr20/E2fNet.
Abstract
While functional magnetic resonance imaging (fMRI) offers valuable insights into brain activity, it is limited by high operational costs and significant infrastructural demands. In contrast, electroencephalography (EEG) provides millisecond-level precision in capturing electrical activity but lacks the spatial fidelity necessary for precise neural localization. To bridge these gaps, we propose E2fNet, a simple yet effective deep learning model for synthesizing fMRI images from low-cost EEG data. E2fNet is an encoder-decoder network specifically designed to capture and translate meaningful multi-scale features from EEG across electrode channels into accurate fMRI representations. Extensive evaluations across three public datasets demonstrate that E2fNet consistently outperforms existing CNN- and transformer-based methods, achieving state-of-the-art results in terms of the structural similarity index measure (SSIM). These results demonstrate that E2fNet is a promising, cost-effective solution for enhancing neuroimaging capabilities. The code is available at https://github.com/kgr20/E2fNet.