Deep sr-DDL: Deep Structurally Regularized Dynamic Dictionary Learning to Integrate Multimodal and Dynamic Functional Connectomics data for Multidimensional Clinical Characterizations

TL;DR

A novel integrated framework that jointly models complementary information from resting-state functional MRI (rs-fMRI) connectivity and diffusion tensor imaging (DTI) tractography to extract biomarkers of brain connectivity predictive of behavior outperforms several state-of-the-art approaches at clinical outcome prediction and learns interpretable multimodal neural signatures of brain organization.

Abstract

We propose a novel integrated framework that jointly models complementary information from resting-state functional MRI (rs-fMRI) connectivity and diffusion tensor imaging (DTI) tractography to extract biomarkers of brain connectivity predictive of behavior. Our framework couples a generative model of the connectomics data with a deep network that predicts behavioral scores. The generative component is a structurally-regularized Dynamic Dictionary Learning (sr-DDL) model that decomposes the dynamic rs-fMRI correlation matrices into a collection of shared basis networks and time varying subject-specific loadings. We use the DTI tractography to regularize this matrix factorization and learn anatomically informed functional connectivity profiles. The deep component of our framework is an LSTM-ANN block, which uses the temporal evolution of the subject-specific sr-DDL loadings to predict multidimensional clinical characterizations. Our joint optimization strategy collectively estimates the basis networks, the subject-specific time-varying loadings, and the neural network weights. We validate our framework on a dataset of neurotypical individuals from the Human Connectome Project (HCP) database to map to cognition and on a separate multi-score prediction task on individuals diagnosed with Autism Spectrum Disorder (ASD) in a five-fold cross validation setting. Our hybrid model outperforms several state-of-the-art approaches at clinical outcome prediction and learns interpretable multimodal neural signatures of brain organization.

Figures (18)

• Figure 1: Top: For the fMRI data, we group voxels in the brain into ROIs defined by a standard atlas and compute the average time courses for each ROI. The correlation matrix captures the synchrony in the average time courses. Bottom Tractography is performed on the raw DWI data to track the path of neuronal fibers in the brain. Based on the parcellation scheme, we construct a map of the fibre tracts between ROIs in the brain. The same parcellation scheme is used for both modalities.
• Figure 2: First, the ROI's defined by a standard atlas are used to compute regional time series. Then, a sliding window protocol defined by window length and stride is applied to extract the dynamic patient correlation matrices. As in the static case, the dynamic matrices measure the synchrony between regional time series, but as a function of time.
• Figure 3: Framework to integrate structural and dynamic functional connectivity for clinical severity prediction Green Box: The generative sr-DDL module. The rs-fMRI dynamic correlation matrices are decomposed into the subnetwork basis and time-varying subject-specific loadings. The DTI connectivity regularizes this decomposition. Purple Box: Deep LSTM-ANN module for multi-score prediction. The sr-DDL coefficients are input into the LSTM to generate a hidden representation. The predictor ANN (P-ANN) generates a time varying estimate for the scores, while the attention ANN (A-ANN) weights the predictions across time to generate the final clinical severity estimate.
• Figure 4: Alternating minimization strategy for joint optimization of Eq. (\ref{['eqn:const']})
• Figure 5: Scree Plot of the correlation matrices to corroborate the selected values for $K$. (L) KKI Dataset (R) HCP Dataset. The thick line denotes the mean eigenvalue, while the shaded area indicates the standard deviation across subjects and time points.