Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multi-modal Imaging Genomics Transformer: Attentive Integration of Imaging with Genomic Biomarkers for Schizophrenia Classification

Nagur Shareef Shaik, Teja Krishna Cherukuri, Vince D. Calhoun, Dong Hye Ye

TL;DR

Schizophrenia diagnosis benefits from integrating structural MRI, functional connectome, and genomic biomarkers, but effective multi‑modal fusion remains challenging. The authors propose MIGTrans, a three‑way attentive transformer that uses Genomic and Connectome Encoders, a Structural MRI encoder with Spatial Sequence Attention, and a Fusion Transformer to perform cross‑modal fusion through stepwise attention. The model achieves an accuracy of $86.05\%$ (±$0.02$) on a FBIRN‑derived dataset and provides interpretable biomarkers by highlighting significant SNPs and pivotal brain regions and connections. This work advances diagnostic performance and offers biological insights that could inform personalized interventions for schizophrenia.

Abstract

Schizophrenia (SZ) is a severe brain disorder marked by diverse cognitive impairments, abnormalities in brain structure, function, and genetic factors. Its complex symptoms and overlap with other psychiatric conditions challenge traditional diagnostic methods, necessitating advanced systems to improve precision. Existing research studies have mostly focused on imaging data, such as structural and functional MRI, for SZ diagnosis. There has been less focus on the integration of genomic features despite their potential in identifying heritable SZ traits. In this study, we introduce a Multi-modal Imaging Genomics Transformer (MIGTrans), that attentively integrates genomics with structural and functional imaging data to capture SZ-related neuroanatomical and connectome abnormalities. MIGTrans demonstrated improved SZ classification performance with an accuracy of 86.05% (+/- 0.02), offering clear interpretations and identifying significant genomic locations and brain morphological/connectivity patterns associated with SZ.

Multi-modal Imaging Genomics Transformer: Attentive Integration of Imaging with Genomic Biomarkers for Schizophrenia Classification

TL;DR

Schizophrenia diagnosis benefits from integrating structural MRI, functional connectome, and genomic biomarkers, but effective multi‑modal fusion remains challenging. The authors propose MIGTrans, a three‑way attentive transformer that uses Genomic and Connectome Encoders, a Structural MRI encoder with Spatial Sequence Attention, and a Fusion Transformer to perform cross‑modal fusion through stepwise attention. The model achieves an accuracy of (±) on a FBIRN‑derived dataset and provides interpretable biomarkers by highlighting significant SNPs and pivotal brain regions and connections. This work advances diagnostic performance and offers biological insights that could inform personalized interventions for schizophrenia.

Abstract

Schizophrenia (SZ) is a severe brain disorder marked by diverse cognitive impairments, abnormalities in brain structure, function, and genetic factors. Its complex symptoms and overlap with other psychiatric conditions challenge traditional diagnostic methods, necessitating advanced systems to improve precision. Existing research studies have mostly focused on imaging data, such as structural and functional MRI, for SZ diagnosis. There has been less focus on the integration of genomic features despite their potential in identifying heritable SZ traits. In this study, we introduce a Multi-modal Imaging Genomics Transformer (MIGTrans), that attentively integrates genomics with structural and functional imaging data to capture SZ-related neuroanatomical and connectome abnormalities. MIGTrans demonstrated improved SZ classification performance with an accuracy of 86.05% (+/- 0.02), offering clear interpretations and identifying significant genomic locations and brain morphological/connectivity patterns associated with SZ.
Paper Structure (16 sections, 12 equations, 2 figures, 1 table)

This paper contains 16 sections, 12 equations, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Multi-modal Imaging Genomics Transformer
  3. Genomic Connectome Representation Learning
  4. Genomic Encoder
  5. Connectome Encoder
  6. Morphological Representation Learning
  7. Structural MRI Encoder
  8. Spatial Sequence Attention
  9. Fusion Transformer
  10. Genomic Connectome TransFusor
  11. Genomic Connectome sMRI TransFusor
  12. Experiments & Results
  13. Dataset, Pre-processing & Experimental Setup
  14. Quantitative Evaluation
  15. Qualitative Evaluation
  16. ...and 1 more sections

Figures (2)

  • Figure 1: Architecture of Multi-modal Imaging Genomics Transformer (MIGTrans) - employing Genomic & Connectome Encoders for genomic & connectome representation learning; Structural MRI encoder and Spatial Sequence Attention for morphological representation learning; Fusion Transformer for multi-modal step-wise attentive integration
  • Figure 2: Attention maps of sMRI highlighting the frontal and temporal lobes in SZ samples, contrasting with thalamic regions in HC samples; Functional brain networks highlighting important connections in default mode (DMN) and cognitive control (CON) specific to SZ and HC compared with other subcortical (SCN), auditory (ADN), sensorimotor (SMN), visual (VSN), and cerebellum (CBN) networks.