Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

NeuroPath: A Neural Pathway Transformer for Joining the Dots of Human Connectomes

Ziquan Wei, Tingting Dan, Jiaqi Ding, Guorong Wu

TL;DR

A biological-inspired deep model is proposed, coined as NeuroPath, to find putative connectomic feature representations from the unprecedented amount of neuroimages, which can be plugged into various downstream applications such as task recognition and disease diagnosis.

Abstract

Although modern imaging technologies allow us to study connectivity between two distinct brain regions in-vivo, an in-depth understanding of how anatomical structure supports brain function and how spontaneous functional fluctuations emerge remarkable cognition is still elusive. Meanwhile, tremendous efforts have been made in the realm of machine learning to establish the nonlinear mapping between neuroimaging data and phenotypic traits. However, the absence of neuroscience insight in the current approaches poses significant challenges in understanding cognitive behavior from transient neural activities. To address this challenge, we put the spotlight on the coupling mechanism of structural connectivity (SC) and functional connectivity (FC) by formulating such network neuroscience question into an expressive graph representation learning problem for high-order topology. Specifically, we introduce the concept of topological detour to characterize how a ubiquitous instance of FC (direct link) is supported by neural pathways (detour) physically wired by SC, which forms a cyclic loop interacted by brain structure and function. In the cliché of machine learning, the multi-hop detour pathway underlying SC-FC coupling allows us to devise a novel multi-head self-attention mechanism within Transformer to capture multi-modal feature representation from paired graphs of SC and FC. Taken together, we propose a biological-inspired deep model, coined as NeuroPath, to find putative connectomic feature representations from the unprecedented amount of neuroimages, which can be plugged into various downstream applications such as task recognition and disease diagnosis. We have evaluated NeuroPath on large-scale public datasets including HCP and UK Biobank under supervised and zero-shot learning, where the state-of-the-art performance by our NeuroPath indicates great potential in network neuroscience.

NeuroPath: A Neural Pathway Transformer for Joining the Dots of Human Connectomes

TL;DR

A biological-inspired deep model is proposed, coined as NeuroPath, to find putative connectomic feature representations from the unprecedented amount of neuroimages, which can be plugged into various downstream applications such as task recognition and disease diagnosis.

Abstract

Although modern imaging technologies allow us to study connectivity between two distinct brain regions in-vivo, an in-depth understanding of how anatomical structure supports brain function and how spontaneous functional fluctuations emerge remarkable cognition is still elusive. Meanwhile, tremendous efforts have been made in the realm of machine learning to establish the nonlinear mapping between neuroimaging data and phenotypic traits. However, the absence of neuroscience insight in the current approaches poses significant challenges in understanding cognitive behavior from transient neural activities. To address this challenge, we put the spotlight on the coupling mechanism of structural connectivity (SC) and functional connectivity (FC) by formulating such network neuroscience question into an expressive graph representation learning problem for high-order topology. Specifically, we introduce the concept of topological detour to characterize how a ubiquitous instance of FC (direct link) is supported by neural pathways (detour) physically wired by SC, which forms a cyclic loop interacted by brain structure and function. In the cliché of machine learning, the multi-hop detour pathway underlying SC-FC coupling allows us to devise a novel multi-head self-attention mechanism within Transformer to capture multi-modal feature representation from paired graphs of SC and FC. Taken together, we propose a biological-inspired deep model, coined as NeuroPath, to find putative connectomic feature representations from the unprecedented amount of neuroimages, which can be plugged into various downstream applications such as task recognition and disease diagnosis. We have evaluated NeuroPath on large-scale public datasets including HCP and UK Biobank under supervised and zero-shot learning, where the state-of-the-art performance by our NeuroPath indicates great potential in network neuroscience.
Paper Structure (30 sections, 1 theorem, 10 equations, 8 figures, 9 tables)

This paper contains 30 sections, 1 theorem, 10 equations, 8 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Motivations
  4. Relevant Machine Learning Work for Brain Connectomes
  5. NeuroPath -- A Biological-Inspired Transformer for Human Connectomes
  6. Problem formulation
  7. Network Architecture of NeuroPath
  8. Neural Pathway Representation by NeuroPath
  9. Experimental Results
  10. Performance of Neural Activity Classification
  11. Performance of Cognition Disordering Diagnosis
  12. Zero-shot Learning
  13. Ablation of Hyperparameter $H$
  14. Ablation of Model Architecture
  15. Ablation of Model Depth
  16. ...and 15 more sections

Key Result

Lemma A.1

The node collection of half of the neural pathways within $H$-hop started at the $i$-th node can be sorted as $\mathbf{P}^H_i\in\{\text{argtrue}\mathbf{D}^h_{i,\cdot}\}_{h=1,\dots,H}$, where 'collection' denotes a set allowing repeated members.

Figures (8)

  • Figure 1: Planting a novel multivariate SC-FC coupling mechanism to explainable deep model. Orange box: The structural connectivity (SC) denoted by grey links represents the strength of neurological fiber that physically connects two brain regions. SC is relatively static given the neural activities are transient, e.g. cognitive tasks. Green box: The functional connectivity (FC) is commonly considered as the brain network topology said2023neurograph since SC is static for different cognitive tasks. The overlapping area of orange and green boxes: the multivariate SC-FC coupling mechanism, where a neural pathway (detour) is constructed by multiple SC links to support one FC link. Grey box: NeuroPath Transformer using a new MHSA module filtered by adjacency matrices emits the representation of multi-hop detours.
  • Figure 2: Motivation of integrating SC-FC coupling in neural networks. Top: Brain regions manifesting significant resting state vs VISMOTOR difference using detour degree and FC degree. Color indicate the $p$-values ($0\leq p \leq0.05$) in paired t-test. Bottom: The overlap ratio (y-axis) between the number of identified significant regions and the total number of brain regions in each pre-defined functional sub-networks (x-axis). Specifically, we examine the identified regions in default mode (red circle/box), visual (orange circle/box), and sensorimotor (green circle/box) networks since they are closely associated with resting stage vs VISMOTOR difference.
  • Figure 3: Framework of twin branch, topological detour filtered multi-head self-attention (TD-MHSA) and functional connectivity filtered multi-head self-attention (FC-MHSA), for node feature transformation in NeuroPath, where training/testing readout indicates different branch is used for training/testing stage.
  • Figure 4: Ablation study of various lengths of the neural pathway that is visible to NeuroPath. Static BOLD is set as node attributes in this experiment. The blue shade is the range of error bars and the green lines are average F1 scores.
  • Figure 5: The visualization of the top-1 neural pathway that corresponds to a significant FC link contributing to the prediction by NeuroPath on OASIS dataset.
  • ...and 3 more figures

Theorems & Definitions (3)

  • Definition 3.1: Detour adjacency matrix
  • proof
  • Lemma A.1