Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

R-GenIMA: Integrating Neuroimaging and Genetics with Interpretable Multimodal AI for Alzheimer's Disease Progression

Kun Zhao, Siyuan Dai, Yingying Zhang, Guodong Liu, Pengfei Gu, Chenghua Lin, Paul M. Thompson, Alex Leow, Heng Huang, Lifang He, Liang Zhan, Haoteng Tang

TL;DR

The paper introduces R-GenIMA, an interpretable multimodal AI that fuses ROI-wise MRI tokens with SNP prompts via a large language model to model Alzheimer's disease progression across NC, SMC, MCI, and AD. By preserving regional anatomical detail with RiT and aligning genetic variation through structured prompts, the approach enables cross-modal reasoning and yields biologically meaningful ROI–gene associations. On the ADNI dataset, R-GenIMA achieves state-of-the-art four-way classification, with the Mixture Data configuration delivering near-perfect accuracy and the model-prioritized genes showing enrichment for established AD risk loci. The results reveal stage-specific neuroanatomical signatures and cohesive ROI–gene patterns that reflect a progression from synaptic vulnerability to network disruption, highlighting the potential for early risk stratification and mechanistic insight while acknowledging generalizability and data-panel limitations.

Abstract

Early detection of Alzheimer's disease (AD) requires models capable of integrating macro-scale neuroanatomical alterations with micro-scale genetic susceptibility, yet existing multimodal approaches struggle to align these heterogeneous signals. We introduce R-GenIMA, an interpretable multimodal large language model that couples a novel ROI-wise vision transformer with genetic prompting to jointly model structural MRI and single nucleotide polymorphisms (SNPs) variations. By representing each anatomically parcellated brain region as a visual token and encoding SNP profiles as structured text, the framework enables cross-modal attention that links regional atrophy patterns to underlying genetic factors. Applied to the ADNI cohort, R-GenIMA achieves state-of-the-art performance in four-way classification across normal cognition (NC), subjective memory concerns (SMC), mild cognitive impairment (MCI), and AD. Beyond predictive accuracy, the model yields biologically meaningful explanations by identifying stage-specific brain regions and gene signatures, as well as coherent ROI-Gene association patterns across the disease continuum. Attention-based attribution revealed genes consistently enriched for established GWAS-supported AD risk loci, including APOE, BIN1, CLU, and RBFOX1. Stage-resolved neuroanatomical signatures identified shared vulnerability hubs across disease stages alongside stage-specific patterns: striatal involvement in subjective decline, frontotemporal engagement during prodromal impairment, and consolidated multimodal network disruption in AD. These results demonstrate that interpretable multimodal AI can synthesize imaging and genetics to reveal mechanistic insights, providing a foundation for clinically deployable tools that enable earlier risk stratification and inform precision therapeutic strategies in Alzheimer's disease.

R-GenIMA: Integrating Neuroimaging and Genetics with Interpretable Multimodal AI for Alzheimer's Disease Progression

TL;DR

The paper introduces R-GenIMA, an interpretable multimodal AI that fuses ROI-wise MRI tokens with SNP prompts via a large language model to model Alzheimer's disease progression across NC, SMC, MCI, and AD. By preserving regional anatomical detail with RiT and aligning genetic variation through structured prompts, the approach enables cross-modal reasoning and yields biologically meaningful ROI–gene associations. On the ADNI dataset, R-GenIMA achieves state-of-the-art four-way classification, with the Mixture Data configuration delivering near-perfect accuracy and the model-prioritized genes showing enrichment for established AD risk loci. The results reveal stage-specific neuroanatomical signatures and cohesive ROI–gene patterns that reflect a progression from synaptic vulnerability to network disruption, highlighting the potential for early risk stratification and mechanistic insight while acknowledging generalizability and data-panel limitations.

Abstract

Early detection of Alzheimer's disease (AD) requires models capable of integrating macro-scale neuroanatomical alterations with micro-scale genetic susceptibility, yet existing multimodal approaches struggle to align these heterogeneous signals. We introduce R-GenIMA, an interpretable multimodal large language model that couples a novel ROI-wise vision transformer with genetic prompting to jointly model structural MRI and single nucleotide polymorphisms (SNPs) variations. By representing each anatomically parcellated brain region as a visual token and encoding SNP profiles as structured text, the framework enables cross-modal attention that links regional atrophy patterns to underlying genetic factors. Applied to the ADNI cohort, R-GenIMA achieves state-of-the-art performance in four-way classification across normal cognition (NC), subjective memory concerns (SMC), mild cognitive impairment (MCI), and AD. Beyond predictive accuracy, the model yields biologically meaningful explanations by identifying stage-specific brain regions and gene signatures, as well as coherent ROI-Gene association patterns across the disease continuum. Attention-based attribution revealed genes consistently enriched for established GWAS-supported AD risk loci, including APOE, BIN1, CLU, and RBFOX1. Stage-resolved neuroanatomical signatures identified shared vulnerability hubs across disease stages alongside stage-specific patterns: striatal involvement in subjective decline, frontotemporal engagement during prodromal impairment, and consolidated multimodal network disruption in AD. These results demonstrate that interpretable multimodal AI can synthesize imaging and genetics to reveal mechanistic insights, providing a foundation for clinically deployable tools that enable earlier risk stratification and inform precision therapeutic strategies in Alzheimer's disease.

Paper Structure

This paper contains 27 sections, 17 equations, 3 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Results
  3. Multimodal Classification Performance Across Alzheimer's Disease Stages
  4. Model-Prioritized Genes Show Significant Enrichment for Established AD Risk Loci
  5. Bootstrap-Validated Brain Regions Reveal Stage-Specific Vulnerability Patterns
  6. Reproducible ROI–Gene Associations Across Disease Progression
  7. Discussion
  8. ROI-Level Decomposition and Semantic Alignment Enable Effective Multimodal Integration
  9. Heterogeneous Training Data Enhances Multimodal Learning and Cross-Category Robustness
  10. Stage-Resolved Neuroanatomical Patterns Align with Established Neurobiology of Early AD
  11. Model-Prioritized Genes Span Classical Risk Loci, Synaptic Regulators, and Neuroimmune Pathways
  12. Brain ROI–Gene Associations Evolve Systematically from Synaptic Vulnerability to Network Disruption
  13. Translational Potential and Current Limitations of the Multimodal Framework
  14. Methods
  15. Preliminary Konwledge
  16. ...and 12 more sections

Figures (3)

  • Figure 1: Identified brain ROIs associated with disease stages. SMC associated brain ROIs are Left-Cerebellum-Cortex, Left-Thalamus, ctx-rh-insula, Left-Caudate, Left-Putamen and ctx-rh-transversetemporal. MCI associated brain ROIs are Left-Cerebellum-Cortex, ctx-rh-insula, ctx-rh-transversetemporal, ctx-rh-temporalpole, ctx-rh-frontalpole and Left-Thalamus. AD associated brain ROIs are Left-Cerebellum-Cortex, ctx-rh-insula, Left-Thalamus, ctx-rh-transversetemporal and Left-Caudate.
  • Figure 2: Identified ROI–Gene association patterns in the SMC, MCI, and AD cohorts. For each cohort, the two genes with the highest stability scores per ROI are annotated.
  • Figure 3: (1) The R-GeneIMA framework integrates ROI-based neuroimaging features with SNPs profiles. ROI patches are embedded using the proposed RiT model, and a structured prompt couples these embeddings with the SNPs profile before being passed to the LLM for stage-specific reasoning. (2) Illustration of the 3D ROI parcellation process. (3) Illustration of the RiT model.