NeuroMambaLLM: Dynamic Graph Learning of fMRI Functional Connectivity in Autistic Brains Using Mamba and Language Model Reasoning

TL;DR

NeuroMambaLLM addresses the limitations of static fMRI analysis by learning time-varying functional connectivity directly from raw BOLD signals and aligning these dynamic graphs with a frozen LLM through LoRA-tuned adapters. The method combines a dynamic latent graph encoder with a linear-time selective state-space backbone (Mamba) and a cross-modal alignment mechanism to enable both ASD classification and text-based reasoning about neural dynamics. Key contributions include: (i) an end-to-end dynamic graph learner that eschews fixed connectivity priors, (ii) efficient long-range temporal modelling suitable for long fMRI sequences, and (iii) brain–LLM alignment via brain-summary tokens that produce clinically grounded textual reports. The approach yields competitive accuracy on ABIDE with interpretable explanations, highlighting potential for clinically useful reasoning beyond black-box predictions, and sets the stage for multimodal extensions and broader generalization across datasets.

Abstract

Large Language Models (LLMs) have demonstrated strong semantic reasoning across multimodal domains. However, their integration with graph-based models of brain connectivity remains limited. In addition, most existing fMRI analysis methods rely on static Functional Connectivity (FC) representations, which obscure transient neural dynamics critical for neurodevelopmental disorders such as autism. Recent state-space approaches, including Mamba, model temporal structure efficiently, but are typically used as standalone feature extractors without explicit high-level reasoning. We propose NeuroMambaLLM, an end-to-end framework that integrates dynamic latent graph learning and selective state-space temporal modelling with LLMs. The proposed method learns the functional connectivity dynamically from raw Blood-Oxygen-Level-Dependent (BOLD) time series, replacing fixed correlation graphs with adaptive latent connectivity while suppressing motion-related artifacts and capturing long-range temporal dependencies. The resulting dynamic brain representations are projected into the embedding space of an LLM model, where the base language model remains frozen and lightweight low-rank adaptation (LoRA) modules are trained for parameter-efficient alignment. This design enables the LLM to perform both diagnostic classification and language-based reasoning, allowing it to analyze dynamic fMRI patterns and generate clinically meaningful textual reports.

Figures (6)

• Figure 1: Ablation results. Comparison of temporal modelling backbones and the effect of integrating an LLM with frozen and LoRA-based adaptation.
• Figure 2: Example LLM prompt and generated clinical response.
• Figure 3: Effect of brain–LLM alignment during fine-tuning. Structured brain-summary tokens improve performance, while mean pooling offers limited gains and random tokens provide no benefit.
• Figure 4: Regional brain activations based on NeuroMambaLLM Results. Prominent involvement of the left superior temporal sulcus (STS) and posterior occipital regions highlights circuits related to social perception/audiovisual integration and early visual-sensory processing.
• Figure 5: Most informative functional connectivity patterns projected onto the cortical surface. From left to right: left lateral, medial, and right lateral views. The model highlights predominantly intra-hemispheric and long-range fronto-temporal connections, with left-hemisphere dominance.