Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Deep-Generative Hybrid Model to Integrate Multimodal and Dynamic Connectivity for Predicting Spectrum-Level Deficits in Autism

Niharika Shimona D'Souza, Mary Beth Nebel, Deana Crocetti, Nicholas Wymbs, Joshua Robinson, Stewart Mostofsky, Archana Venkataraman

TL;DR

The integrated deep-generative framework outperforms state-of-the-art baselines in a five-fold cross validated setting and extracts interpretable multimodal neural signatures of brain dysfunction in ASD.

Abstract

We propose an integrated deep-generative framework, that jointly models complementary information from resting-state functional MRI (rs-fMRI) connectivity and diffusion tensor imaging (DTI) tractography to extract predictive biomarkers of a disease. The generative part of our framework is a structurally-regularized Dynamic Dictionary Learning (sr-DDL) model that decomposes the dynamic rs-fMRI correlation matrices into a collection of shared basis networks and time varying patient-specific loadings. This matrix factorization is guided by the DTI tractography matrices to learn anatomically informed connectivity profiles. The deep part of our framework is an LSTM-ANN block, which models the temporal evolution of the patient sr-DDL loadings to predict multidimensional clinical severity. Our coupled optimization procedure collectively estimates the basis networks, the patient-specific dynamic loadings, and the neural network weights. We validate our framework on a multi-score prediction task in 57 patients diagnosed with Autism Spectrum Disorder (ASD). Our hybrid model outperforms state-of-the-art baselines in a five-fold cross validated setting and extracts interpretable multimodal neural signatures of brain dysfunction in ASD.

A Deep-Generative Hybrid Model to Integrate Multimodal and Dynamic Connectivity for Predicting Spectrum-Level Deficits in Autism

TL;DR

The integrated deep-generative framework outperforms state-of-the-art baselines in a five-fold cross validated setting and extracts interpretable multimodal neural signatures of brain dysfunction in ASD.

Abstract

We propose an integrated deep-generative framework, that jointly models complementary information from resting-state functional MRI (rs-fMRI) connectivity and diffusion tensor imaging (DTI) tractography to extract predictive biomarkers of a disease. The generative part of our framework is a structurally-regularized Dynamic Dictionary Learning (sr-DDL) model that decomposes the dynamic rs-fMRI correlation matrices into a collection of shared basis networks and time varying patient-specific loadings. This matrix factorization is guided by the DTI tractography matrices to learn anatomically informed connectivity profiles. The deep part of our framework is an LSTM-ANN block, which models the temporal evolution of the patient sr-DDL loadings to predict multidimensional clinical severity. Our coupled optimization procedure collectively estimates the basis networks, the patient-specific dynamic loadings, and the neural network weights. We validate our framework on a multi-score prediction task in 57 patients diagnosed with Autism Spectrum Disorder (ASD). Our hybrid model outperforms state-of-the-art baselines in a five-fold cross validated setting and extracts interpretable multimodal neural signatures of brain dysfunction in ASD.

Paper Structure

This paper contains 6 sections, 6 equations, 4 figures, 1 table.

Table of Contents

  1. Introduction
  2. A Deep-Generative Hybrid Model for Connectomics
  3. Coupled Optimization Strategy:
  4. Baseline Comparisons:
  5. Experimental Evaluation and Results:
  6. Conclusion

Figures (4)

  • Figure 1: Framework to integrate structural & dynamic functional connectivity for multi-task prediction Gray Box: sr-DDL module for rs-fMRI dynamic correlation matrices and DTI connectivity matrices. Blue Box: LSTM-ANN for multi-score prediction.
  • Figure 2: Alternating minimization strategy for joint optimization of Eq. (\ref{['const']})
  • Figure 3: Multi-Score Prediction by Left: ADOS Middle: SRS Right: Praxis by Yellow Box: deep-generative hybrid. Green Box: PCA+LSTM-ANN
  • Figure 4: Top: Subnetworks identified by the deep-generative hybrid. The red and orange regions are anti-correlated with the blue and green regions Bottom:(Left) Learned attention weights (Right) Variation of network strength over time