Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

BrainPrompt: Multi-Level Brain Prompt Enhancement for Neurological Condition Identification

Jiaxing Xu, Kai He, Yue Tang, Wei Li, Mengcheng Lan, Xia Dong, Yiping Ke, Mengling Feng

TL;DR

BrainPrompt tackles the challenge of diagnosing neurological conditions by enriching graph-based brain-network analysis with LLM-derived, knowledge-driven prompts. By introducing three levels of prompts—ROI-level, subject-level, and disease-level—the framework integrates non-imaging information and external knowledge into GNNs, using a frozen text encoder to map prompts into informative embeddings. Empirical results on ABIDE and ADNI resting-state fMRI datasets show BrainPrompt outperforms multiple baselines, with clear gains from prompt combinations and interpretable biomarker patterns identified via integrated gradients. This approach improves diagnostic accuracy and interpretability, highlighting the potential of LLM-guided, multi-modal prompts for clinical brain network analysis.

Abstract

Neurological conditions, such as Alzheimer's Disease, are challenging to diagnose, particularly in the early stages where symptoms closely resemble healthy controls. Existing brain network analysis methods primarily focus on graph-based models that rely solely on imaging data, which may overlook important non-imaging factors and limit the model's predictive power and interpretability. In this paper, we present BrainPrompt, an innovative framework that enhances Graph Neural Networks (GNNs) by integrating Large Language Models (LLMs) with knowledge-driven prompts, enabling more effective capture of complex, non-imaging information and external knowledge for neurological disease identification. BrainPrompt integrates three types of knowledge-driven prompts: (1) ROI-level prompts to encode the identity and function of each brain region, (2) subject-level prompts that incorporate demographic information, and (3) disease-level prompts to capture the temporal progression of disease. By leveraging these multi-level prompts, BrainPrompt effectively harnesses knowledge-enhanced multi-modal information from LLMs, enhancing the model's capability to predict neurological disease stages and meanwhile offers more interpretable results. We evaluate BrainPrompt on two resting-state functional Magnetic Resonance Imaging (fMRI) datasets from neurological disorders, showing its superiority over state-of-the-art methods. Additionally, a biomarker study demonstrates the framework's ability to extract valuable and interpretable information aligned with domain knowledge in neuroscience. The code is available at https://github.com/AngusMonroe/BrainPrompt

BrainPrompt: Multi-Level Brain Prompt Enhancement for Neurological Condition Identification

TL;DR

BrainPrompt tackles the challenge of diagnosing neurological conditions by enriching graph-based brain-network analysis with LLM-derived, knowledge-driven prompts. By introducing three levels of prompts—ROI-level, subject-level, and disease-level—the framework integrates non-imaging information and external knowledge into GNNs, using a frozen text encoder to map prompts into informative embeddings. Empirical results on ABIDE and ADNI resting-state fMRI datasets show BrainPrompt outperforms multiple baselines, with clear gains from prompt combinations and interpretable biomarker patterns identified via integrated gradients. This approach improves diagnostic accuracy and interpretability, highlighting the potential of LLM-guided, multi-modal prompts for clinical brain network analysis.

Abstract

Neurological conditions, such as Alzheimer's Disease, are challenging to diagnose, particularly in the early stages where symptoms closely resemble healthy controls. Existing brain network analysis methods primarily focus on graph-based models that rely solely on imaging data, which may overlook important non-imaging factors and limit the model's predictive power and interpretability. In this paper, we present BrainPrompt, an innovative framework that enhances Graph Neural Networks (GNNs) by integrating Large Language Models (LLMs) with knowledge-driven prompts, enabling more effective capture of complex, non-imaging information and external knowledge for neurological disease identification. BrainPrompt integrates three types of knowledge-driven prompts: (1) ROI-level prompts to encode the identity and function of each brain region, (2) subject-level prompts that incorporate demographic information, and (3) disease-level prompts to capture the temporal progression of disease. By leveraging these multi-level prompts, BrainPrompt effectively harnesses knowledge-enhanced multi-modal information from LLMs, enhancing the model's capability to predict neurological disease stages and meanwhile offers more interpretable results. We evaluate BrainPrompt on two resting-state functional Magnetic Resonance Imaging (fMRI) datasets from neurological disorders, showing its superiority over state-of-the-art methods. Additionally, a biomarker study demonstrates the framework's ability to extract valuable and interpretable information aligned with domain knowledge in neuroscience. The code is available at https://github.com/AngusMonroe/BrainPrompt

Paper Structure

This paper contains 13 sections, 3 equations, 2 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Problem Definition
  3. Proposed Method
  4. Message-Passing with ROI Prompt Enhancement
  5. Subject-Prompted Population Graph
  6. Temporal Constraint with Disease-Prompt
  7. Experiments
  8. Datasets
  9. Experimental Setup
  10. Main Results
  11. Ablation Study
  12. Biomarker Detection
  13. Conclusion

Figures (2)

  • Figure 1: The architecture of BrainPrompt, using Alzheimer's Disease (AD) as an example. To extract more discriminative information, we do not utilize the entire representation of the used prompts for subsequent computations. Instead, we focus on the portion of tokens of each prompt that differ from other instances, which we define as informative tokens (denoted by red color).
  • Figure 2: Visualization of the top 10 salient ROIs according to the integrated gradients of BrainPrompt-G (the magnitude of saliency is represented by color intensity). We also show some of the corresponding ROI-level prompts.