Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Modeling Spatiotemporal Neural Frames for High Resolution Brain Dynamic

Wanying Qu, Jianxiong Gao, Wei Wang, Yanwei Fu

Abstract

Capturing dynamic spatiotemporal neural activity is essential for understanding large-scale brain mechanisms. Functional magnetic resonance imaging (fMRI) provides high-resolution cortical representations that form a strong basis for characterizing fine-grained brain activity patterns. The high acquisition cost of fMRI limits large-scale applications, therefore making high-quality fMRI reconstruction a crucial task. Electroencephalography (EEG) offers millisecond-level temporal cues that complement fMRI. Leveraging this complementarity, we present an EEG-conditioned framework for reconstructing dynamic fMRI as continuous neural sequences with high spatial fidelity and strong temporal coherence at the cortical-vertex level. To address sampling irregularities common in real fMRI acquisitions, we incorporate a null-space intermediate-frame reconstruction, enabling measurement-consistent completion of arbitrary intermediate frames and improving sequence continuity and practical applicability. Experiments on the CineBrain dataset demonstrate superior voxel-wise reconstruction quality and robust temporal consistency across whole-brain and functionally specific regions. The reconstructed fMRI also preserves essential functional information, supporting downstream visual decoding tasks. This work provides a new pathway for estimating high-resolution fMRI dynamics from EEG and advances multimodal neuroimaging toward more dynamic brain activity modeling.

Modeling Spatiotemporal Neural Frames for High Resolution Brain Dynamic

Abstract

Capturing dynamic spatiotemporal neural activity is essential for understanding large-scale brain mechanisms. Functional magnetic resonance imaging (fMRI) provides high-resolution cortical representations that form a strong basis for characterizing fine-grained brain activity patterns. The high acquisition cost of fMRI limits large-scale applications, therefore making high-quality fMRI reconstruction a crucial task. Electroencephalography (EEG) offers millisecond-level temporal cues that complement fMRI. Leveraging this complementarity, we present an EEG-conditioned framework for reconstructing dynamic fMRI as continuous neural sequences with high spatial fidelity and strong temporal coherence at the cortical-vertex level. To address sampling irregularities common in real fMRI acquisitions, we incorporate a null-space intermediate-frame reconstruction, enabling measurement-consistent completion of arbitrary intermediate frames and improving sequence continuity and practical applicability. Experiments on the CineBrain dataset demonstrate superior voxel-wise reconstruction quality and robust temporal consistency across whole-brain and functionally specific regions. The reconstructed fMRI also preserves essential functional information, supporting downstream visual decoding tasks. This work provides a new pathway for estimating high-resolution fMRI dynamics from EEG and advances multimodal neuroimaging toward more dynamic brain activity modeling.
Paper Structure (17 sections, 10 equations, 7 figures, 6 tables, 1 algorithm)

This paper contains 17 sections, 10 equations, 7 figures, 6 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. EEG-to-fMRI Translation
  4. Diffusion Models
  5. Method
  6. Task Formulation
  7. Denoising Null-Space Diffusion Transformer
  8. Model Architecture
  9. Training & Inference
  10. Experiments
  11. Setting
  12. Result and Analysis
  13. Conclusion and Discussion
  14. Data Preprocessing
  15. Subject-wise Dynamic fMRI Reconstruction
  16. ...and 2 more sections

Figures (7)

  • Figure 1: This paper reconstructs dynamic fMRI frames with high spatial detail and temporal coherence from EEG. We further introduce intermediate frame reconstruction (InterRecon), and use visual decoding as a downstream task to evaluate high-level semantic preservation.
  • Figure 2: Architecture and inference workflow of our framework. Top: End-to-end inference pipeline, where EEG features condition a diffusion-transformer-based reconstruction process to produce spatiotemporal fMRI sequences. Bottom: (a) Transformer-based denoising network that jointly models vertex-level tokens with sequence-level EEG cross-attention to reconstruct coherent fMRI trajectories. (b) Null-space sampling module (InterRecon) that enforces measurement consistency during reverse diffusion, enabling intermediate frame reconstruction without retraining.
  • Figure 3: Comparison of intermediate fMRI frame reconstruction (InterRecon) performance across different frame lengths and methods.
  • Figure 4: Vertex-level cortical surface visualization of reconstructed fMRI alongside ground truth. The top row shows our reconstructed fMRI (Recon.); the middle row shows the ground-truth fMRI (GT); and the bottom row displays the reconstruction error map; the color scale indicates BOLD signal intensity.
  • Figure 5: Functional validation through visual decoding. Comparison between video frames decoded from our reconstructed fMRI (Pred.) and the corresponding ground-truth video frames (GT) using the CineSync-f decoder.
  • ...and 2 more figures