Ensemble manifold based regularized multi-modal graph convolutional network for cognitive ability prediction

TL;DR

This work tackles cognitive ability prediction from multi-modal fMRI by introducing an interpretable ensemble graph convolutional network (MGCN) that fuses ROI time-series and functional connectivity across modalities. The model learns per-modality graph embeddings with a two-layer GCN, then fuses them via an MLP, while a manifold-based regularization enforces subject similarities within and between modalities. Interpretability is achieved through Grad-RAM at the ROI level and an edge mask learning mechanism at the FC level, enabling robust biomarker identification. On the Philadelphia Neurodevelopmental Cohort, MGCN outperforms single-modality GCNs and other baselines in WRAT score prediction and reveals cognition-related biomarkers aligned with established brain networks. The framework thus offers a scalable, interpretable approach for multi-modal brain connectivity analysis and cognitive trait prediction.

Abstract

Objective: Multi-modal functional magnetic resonance imaging (fMRI) can be used to make predictions about individual behavioral and cognitive traits based on brain connectivity networks. Methods: To take advantage of complementary information from multi-modal fMRI, we propose an interpretable multi-modal graph convolutional network (MGCN) model, incorporating the fMRI time series and the functional connectivity (FC) between each pair of brain regions. Specifically, our model learns a graph embedding from individual brain networks derived from multi-modal data. A manifold-based regularization term is then enforced to consider the relationships of subjects both within and between modalities. Furthermore, we propose the gradient-weighted regression activation mapping (Grad-RAM) and the edge mask learning to interpret the model, which is used to identify significant cognition-related biomarkers. Results: We validate our MGCN model on the Philadelphia Neurodevelopmental Cohort to predict individual wide range achievement test (WRAT) score. Our model obtains superior predictive performance over GCN with a single modality and other competing approaches. The identified biomarkers are cross-validated from different approaches. Conclusion and Significance: This paper develops a new interpretable graph deep learning framework for cognitive ability prediction, with the potential to overcome the limitations of several current data-fusion models. The results demonstrate the power of MGCN in analyzing multi-modal fMRI and discovering significant biomarkers for human brain studies.

Figures (6)

• Figure 1: The flowchart of our proposed framework. The GCN block can learn the graph embeddings from multi-modal. Afterward, the manifold based regularization is calculated to incorporate the relations of subjects within and between modalities. The dense layers fuse the embeddings from different modalities for prediction.
• Figure 2: The scatter plot of WRAT score versus age
• Figure 3: The sparsity of edge mask with respect to the $L_{1}$ regularization parameter
• Figure 4: The edge masks learned with different $L_{1}$ regularization parameters
• Figure 5: Module allegiance matrices with different $L_{1}$ regularization parameters