Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Graph Neural Networks with Transformer Fusion of Brain Connectivity Dynamics and Tabular Data for Forecasting Future Tobacco Use

Runzhi Zhou, Xi Luo

TL;DR

This work addresses forecasting future tobacco use by jointly modeling longitudinal resting-state brain connectivity and tabular covariates. It introduces GNN-TF, a time-aware transformer fusion framework that fuses dynamic graph representations from a GNN with structured data, leveraging temporal order to improve prediction. Across NCANDA data, GNN-TF variants outperform state-of-the-art baselines, including GC-LSTM, with robust performance across atlases and through extensive ablations. The approach demonstrates the value of end-to-end fusion of non-Euclidean brain networks and Euclidean clinical data for longitudinal outcome prediction in neuroimaging.

Abstract

Integrating non-Euclidean brain imaging data with Euclidean tabular data, such as clinical and demographic information, poses a substantial challenge for medical imaging analysis, particularly in forecasting future outcomes. While machine learning and deep learning techniques have been applied successfully to cross-sectional classification and prediction tasks, effectively forecasting outcomes in longitudinal imaging studies remains challenging. To address this challenge, we introduce a time-aware graph neural network model with transformer fusion (GNN-TF). This model flexibly integrates both tabular data and dynamic brain connectivity data, leveraging the temporal order of these variables within a coherent framework. By incorporating non-Euclidean and Euclidean sources of information from a longitudinal resting-state fMRI dataset from the National Consortium on Alcohol and Neurodevelopment in Adolescence (NCANDA), the GNN-TF enables a comprehensive analysis that captures critical aspects of longitudinal imaging data. Comparative analyses against a variety of established machine learning and deep learning models demonstrate that GNN-TF outperforms these state-of-the-art methods, delivering superior predictive accuracy for predicting future tobacco usage. The end-to-end, time-aware transformer fusion structure of the proposed GNN-TF model successfully integrates multiple data modalities and leverages temporal dynamics, making it a valuable analytic tool for functional brain imaging studies focused on clinical outcome prediction.

Graph Neural Networks with Transformer Fusion of Brain Connectivity Dynamics and Tabular Data for Forecasting Future Tobacco Use

TL;DR

This work addresses forecasting future tobacco use by jointly modeling longitudinal resting-state brain connectivity and tabular covariates. It introduces GNN-TF, a time-aware transformer fusion framework that fuses dynamic graph representations from a GNN with structured data, leveraging temporal order to improve prediction. Across NCANDA data, GNN-TF variants outperform state-of-the-art baselines, including GC-LSTM, with robust performance across atlases and through extensive ablations. The approach demonstrates the value of end-to-end fusion of non-Euclidean brain networks and Euclidean clinical data for longitudinal outcome prediction in neuroimaging.

Abstract

Integrating non-Euclidean brain imaging data with Euclidean tabular data, such as clinical and demographic information, poses a substantial challenge for medical imaging analysis, particularly in forecasting future outcomes. While machine learning and deep learning techniques have been applied successfully to cross-sectional classification and prediction tasks, effectively forecasting outcomes in longitudinal imaging studies remains challenging. To address this challenge, we introduce a time-aware graph neural network model with transformer fusion (GNN-TF). This model flexibly integrates both tabular data and dynamic brain connectivity data, leveraging the temporal order of these variables within a coherent framework. By incorporating non-Euclidean and Euclidean sources of information from a longitudinal resting-state fMRI dataset from the National Consortium on Alcohol and Neurodevelopment in Adolescence (NCANDA), the GNN-TF enables a comprehensive analysis that captures critical aspects of longitudinal imaging data. Comparative analyses against a variety of established machine learning and deep learning models demonstrate that GNN-TF outperforms these state-of-the-art methods, delivering superior predictive accuracy for predicting future tobacco usage. The end-to-end, time-aware transformer fusion structure of the proposed GNN-TF model successfully integrates multiple data modalities and leverages temporal dynamics, making it a valuable analytic tool for functional brain imaging studies focused on clinical outcome prediction.
Paper Structure (16 sections, 1 equation, 4 figures, 2 tables)

This paper contains 16 sections, 1 equation, 4 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Materials and Methods
  3. Dataset and Preprocessing
  4. Method Overview
  5. Functional Connectivity Construction
  6. Graph Neural Network
  7. Transformer Fusion
  8. Experiments and Results
  9. Experimental Setup
  10. Performance Evaluation
  11. Ablation Studies
  12. Discussion
  13. General Discussion
  14. Contributions of Node Features and Connectivities
  15. Limitations and Future Works
  16. ...and 1 more sections

Figures (4)

  • Figure A1: The proposed GNN-TF network structure for fMRI imaging and structured data in classification tasks. A classification token "cls" is added to the beginning of the sequence in all transformer models as a prompt, except for GPT2. In GPT2, following the guidelines in the OpenAI GPT2 documentation from the Python package "transformers", "cls" is placed at the end of the sequence, with sex and age features being projected to the penultimate position.
  • Figure A2: In ablation studies, an alternative late-fusion strategy (inside the light green box above) replace the TF-fusion module (light blue) in Fig. \ref{['fig:workflow']}.
  • Figure A3: Top five node features ranked by GNNExplainer. Feature importance values range from 0 (no impact on prediction) to 1 (the most impactful)
  • Figure A4: Top 25 brain connections ranked by GNNExplainer. The brain connections are shown in yellow, and the associated brain nodes are shwon in blue.