Hierarchical Spatio-Temporal State-Space Modeling for fMRI Analysis

TL;DR

The paper tackles the challenge of extracting meaningful biomarkers from high-dimensional, dynamic fMRI functional network connectivity (dFNC) by introducing Functional Spatio-Temporal Mamba (FST-Mamba). This architecture uses separate Mamba-based encoders for spatial and temporal information, a component-wise varied-scale aggregation (CVA) to capture inter-network connectivity, and Symmetric Rotary Position Encoding (SymRope) to respect the FNC matrix's symmetry. Bidirectional state-space processing and a component-specific selective scan enhance long-range dependencies while preserving per-component signals. Empirical results on HCP, UKB, and ADNI demonstrate superior classification performance and offer interpretable biomarkers, underlining the value of attention-free sequence modeling for brain discovery. The work provides open-source code to facilitate further exploration and extension.

Abstract

Recent advances in deep learning structured state space models, especially the Mamba architecture, have demonstrated remarkable performance improvements while maintaining linear complexity. In this study, we introduce functional spatiotemporal Mamba (FST-Mamba), a Mamba-based model designed for discovering neurological biomarkers using functional magnetic resonance imaging (fMRI). We focus on dynamic functional network connectivity (dFNC) derived from fMRI and propose a hierarchical spatiotemporal Mamba-based network that processes spatial and temporal information separately using Mamba-based encoders. Leveraging the topological uniqueness of the FNC matrix, we introduce a component-wise varied-scale aggregation (CVA) mechanism to aggregate connectivity across individual components within brain networks, enabling the model to capture component-level and network-level information. Additionally, we propose symmetric rotary position encoding (SymRope) to encode the relative positions of each functional connection while considering the symmetric nature of the FNC matrix. Experimental results demonstrate significant improvements in the proposed FST-Mamba model on various brain-based classification and regression tasks. We further show brain connectivities and dynamics that are crucial for the prediction. Our work reveals the substantial potential of attention-free sequence modeling in brain discovery. The codes are publicly available here: https://github.com/yuxiangwei0808/FunctionalMamba/tree/main.

Figures (4)

• Figure 1: (a) The overall architecture of the proposed FST-Mamba. (b) The fundamental building block of FST-Mamba. Instead of processing space and time simultaneously, FST-Mamba applied separate branches to reduce computational costs while effectively extracting the dynamics of fMRI. (c) FST-Mamba processes the sequence in both forward and backward directions. Additionally, the connectivity mamba encoder further processes the projected sequence in a component-specific order.
• Figure 2: The forward, backward, and the proposed component-specific scan order
• Figure 3: Crucial connections for sex classification (HCP and UKB datasets) and dementia classification (ADNI dataset)
• Figure 4: Periodograms for sex classification (Male vs. Female) and dementia classification (CN vs. AD). Note that the frequency is normalized.