Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

BrainCast: A Spatio-Temporal Forecasting Model for Whole-Brain fMRI Time Series Prediction

Yunlong Gao, Jinbo Yang, Li Xiao, Haiye Huo, Yang Ji, Hao Wang, Aiying Zhang, Yu-Ping Wang

Abstract

Functional magnetic resonance imaging (fMRI) enables noninvasive investigation of brain function, while short clinical scan durations, arising from human and non-human factors, usually lead to reduced data quality and limited statistical power for neuroimaging research. In this paper, we propose BrainCast, a novel spatio-temporal forecasting framework specifically tailored for whole-brain fMRI time series forecasting, to extend informative fMRI time series without additional data acquisition. It formulates fMRI time series forecasting as a multivariate time series prediction task and jointly models temporal dynamics within regions of interest (ROIs) and spatial interactions across ROIs. Specifically, BrainCast integrates a Spatial Interaction Awareness module to characterize inter-ROI dependencies via embedding every ROI time series as a token, a Temporal Feature Refinement module to capture intrinsic neural dynamics within each ROI by enhancing both low- and high-energy temporal components of fMRI time series at the ROI level, and a Spatio-temporal Pattern Alignment module to combine spatial and temporal representations for producing informative whole-brain features. Experimental results on resting-state and task fMRI datasets from the Human Connectome Project demonstrate the superiority of BrainCast over state-of-the-art time series forecasting baselines. Moreover, fMRI time series extended by BrainCast improve downstream cognitive ability prediction, highlighting the clinical and neuroscientific impact brought by whole-brain fMRI time series forecasting in scenarios with restricted scan durations.

BrainCast: A Spatio-Temporal Forecasting Model for Whole-Brain fMRI Time Series Prediction

Abstract

Functional magnetic resonance imaging (fMRI) enables noninvasive investigation of brain function, while short clinical scan durations, arising from human and non-human factors, usually lead to reduced data quality and limited statistical power for neuroimaging research. In this paper, we propose BrainCast, a novel spatio-temporal forecasting framework specifically tailored for whole-brain fMRI time series forecasting, to extend informative fMRI time series without additional data acquisition. It formulates fMRI time series forecasting as a multivariate time series prediction task and jointly models temporal dynamics within regions of interest (ROIs) and spatial interactions across ROIs. Specifically, BrainCast integrates a Spatial Interaction Awareness module to characterize inter-ROI dependencies via embedding every ROI time series as a token, a Temporal Feature Refinement module to capture intrinsic neural dynamics within each ROI by enhancing both low- and high-energy temporal components of fMRI time series at the ROI level, and a Spatio-temporal Pattern Alignment module to combine spatial and temporal representations for producing informative whole-brain features. Experimental results on resting-state and task fMRI datasets from the Human Connectome Project demonstrate the superiority of BrainCast over state-of-the-art time series forecasting baselines. Moreover, fMRI time series extended by BrainCast improve downstream cognitive ability prediction, highlighting the clinical and neuroscientific impact brought by whole-brain fMRI time series forecasting in scenarios with restricted scan durations.
Paper Structure (20 sections, 13 equations, 7 figures, 3 tables)

This paper contains 20 sections, 13 equations, 7 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Time Series Forecasting
  4. Brain fMRI Data Analysis
  5. Methods
  6. Problem Definition
  7. Spatial Interaction Awareness
  8. Temporal Feature Refinement
  9. Spatio-Temporal Pattern Alignment
  10. Forecasting Head and Loss Function
  11. Experimental Results
  12. Datasets and Preprocessing
  13. Implementation Details
  14. Forecasting Performance
  15. Ablation Studies
  16. ...and 5 more sections

Figures (7)

  • Figure 1: The overall framework of BrainCast. (a) fMRI data preprocessing; (b) The Spatial Interaction Awareness (SIA) module for learning between-ROI spatial interactions; (c) The Temporal Feature Refinement (TFR) module for extracting temporal dy- namics within every ROI; and (d) The Spatio-temporal Pattern Alignment (SPA) module for aligning spatio-temporal represent- ations across ROIs and time points.
  • Figure 2: A sliding window partition technique on the fMRI data to generate sequential samples.
  • Figure 3: (a) Illustration of SIAformer, in which self-attention (b) is applied to learning spatial interactions between ROIs.
  • Figure 4: Visualization of forecasting results on a randomly selected ROI of one test sample for HCP-rs-fMRI (a) and HCP-t-fMRI (b), respectively. The black dots denote the previous and ground-truth future data, and the blue dots denote the predicted data.
  • Figure 5: Ablation results for the SIA, TFR, and SPA modules in our BrainCast on both HCP-rs-fMRI (a) and HCP-t-fMRI (b), respectively. The dark blue bars correspond to the left y-axis, whereas the dark red bars correspond to the right y-axis.
  • ...and 2 more figures