DSAM: A Deep Learning Framework for Analyzing Temporal and Spatial Dynamics in Brain Networks

TL;DR

This work tackles the limitation of static or sliding-window functional connectivity in rs-fMRI by learning a goal-specific connectivity matrix directly from time series. The proposed DSAM framework combines temporal convolutional networks, temporal and spatial attention, and ROI-aware graph networks to capture both temporal and spatial brain dynamics in an end-to-end, interpretable manner. Across HCP and ABCD datasets, DSAM outperforms baselines and reveals task-specific connectivity patterns, including sex-related differences, while enabling insight into which time points and ROIs drive predictions. The approach offers a principled pathway to understanding how the brain adapts its functional connectivity to different cognitive or demographic factors, with potential for biomarker discovery and personalized neuroscience.

Abstract

Resting-state functional magnetic resonance imaging (rs-fMRI) is a noninvasive technique pivotal for understanding human neural mechanisms of intricate cognitive processes. Most rs-fMRI studies compute a single static functional connectivity matrix across brain regions of interest, or dynamic functional connectivity matrices with a sliding window approach. These approaches are at risk of oversimplifying brain dynamics and lack proper consideration of the goal at hand. While deep learning has gained substantial popularity for modeling complex relational data, its application to uncovering the spatiotemporal dynamics of the brain is still limited. We propose a novel interpretable deep learning framework that learns goal-specific functional connectivity matrix directly from time series and employs a specialized graph neural network for the final classification. Our model, DSAM, leverages temporal causal convolutional networks to capture the temporal dynamics in both low- and high-level feature representations, a temporal attention unit to identify important time points, a self-attention unit to construct the goal-specific connectivity matrix, and a novel variant of graph neural network to capture the spatial dynamics for downstream classification. To validate our approach, we conducted experiments on the Human Connectome Project dataset with 1075 samples to build and interpret the model for the classification of sex group, and the Adolescent Brain Cognitive Development Dataset with 8520 samples for independent testing. Compared our proposed framework with other state-of-art models, results suggested this novel approach goes beyond the assumption of a fixed connectivity matrix and provides evidence of goal-specific brain connectivity patterns, which opens up the potential to gain deeper insights into how the human brain adapts its functional connectivity specific to the task at hand.

Figures (4)

• Figure 1: Overall Architecture of DSAM . 3 TCN blocks are used to extract the temporal features with different levels of abstraction (low level, medium level, and high level). Temporal attention uses a shared multi head attention module to filter the important time points. Self Attention Block is used to learn the directed FNC matrix, which is used as an input to the Graph Block, followed by Graph Readout and a fully connected layer for classification.
• Figure 2: Regions of significant differences between male and female (HCP)
• Figure 3: (a) Comparison of learned directed FC between males and females, highlighting regions of interest. (b) Visualization of the direction of significance of nodes concerning sex differences.
• Figure 4: Significant nodes within each network (HCP)