Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Large Language Model-based FMRI Encoding of Language Functions for Subjects with Neurocognitive Disorder

Yuejiao Wang, Xianmin Gong, Lingwei Meng, Xixin Wu, Helen Meng

TL;DR

The paper addresses early detection of neurocognitive disorder-related language changes by applying LLM-based fMRI encoding to older adults, a population rarely studied with such models. It builds an encoding framework using LlaMA2-Cantonese context embeddings and voxel-wise ridge regression to derive brain scores that relate contextual language representations to fMRI signals. The study finds that higher MoCA scores correspond to stronger brain scores, with the strongest brain–cognition associations in the middle temporal gyrus (MTG) and related language ROIs, and shows modest or negligible gains from Cantonese-specific pretraining. This work demonstrates the potential of interpretable, language-model-driven fMRI encoders for early NCD detection and highlights the MTG, AG, and SFG as critical regions for language processing in aging and pathology.

Abstract

Functional magnetic resonance imaging (fMRI) is essential for developing encoding models that identify functional changes in language-related brain areas of individuals with Neurocognitive Disorders (NCD). While large language model (LLM)-based fMRI encoding has shown promise, existing studies predominantly focus on healthy, young adults, overlooking older NCD populations and cognitive level correlations. This paper explores language-related functional changes in older NCD adults using LLM-based fMRI encoding and brain scores, addressing current limitations. We analyze the correlation between brain scores and cognitive scores at both whole-brain and language-related ROI levels. Our findings reveal that higher cognitive abilities correspond to better brain scores, with correlations peaking in the middle temporal gyrus. This study highlights the potential of fMRI encoding models and brain scores for detecting early functional changes in NCD patients.

Large Language Model-based FMRI Encoding of Language Functions for Subjects with Neurocognitive Disorder

TL;DR

The paper addresses early detection of neurocognitive disorder-related language changes by applying LLM-based fMRI encoding to older adults, a population rarely studied with such models. It builds an encoding framework using LlaMA2-Cantonese context embeddings and voxel-wise ridge regression to derive brain scores that relate contextual language representations to fMRI signals. The study finds that higher MoCA scores correspond to stronger brain scores, with the strongest brain–cognition associations in the middle temporal gyrus (MTG) and related language ROIs, and shows modest or negligible gains from Cantonese-specific pretraining. This work demonstrates the potential of interpretable, language-model-driven fMRI encoders for early NCD detection and highlights the MTG, AG, and SFG as critical regions for language processing in aging and pathology.

Abstract

Functional magnetic resonance imaging (fMRI) is essential for developing encoding models that identify functional changes in language-related brain areas of individuals with Neurocognitive Disorders (NCD). While large language model (LLM)-based fMRI encoding has shown promise, existing studies predominantly focus on healthy, young adults, overlooking older NCD populations and cognitive level correlations. This paper explores language-related functional changes in older NCD adults using LLM-based fMRI encoding and brain scores, addressing current limitations. We analyze the correlation between brain scores and cognitive scores at both whole-brain and language-related ROI levels. Our findings reveal that higher cognitive abilities correspond to better brain scores, with correlations peaking in the middle temporal gyrus. This study highlights the potential of fMRI encoding models and brain scores for detecting early functional changes in NCD patients.
Paper Structure (12 sections, 1 equation, 4 figures, 2 tables)

This paper contains 12 sections, 1 equation, 4 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Data Collection
  3. Approach
  4. LlaMA2-Cantonese and Context Features
  5. Encoding Model and Hyperparameter Selection
  6. Experimental Design
  7. Results and Discussions
  8. Brain Score Analysis of the Whole Brain
  9. Brain Score Analysis in Language-related ROIs
  10. Effect of Cantonese Pretraining
  11. Conclusion
  12. Acknowledgements

Figures (4)

  • Figure 1: Construction of a language encoding model based on the movie-watching task, and the correlation between brain scores and MoCA scores.
  • Figure 2: Stimulus matrix construction with time alignment, feature dimension reduction and finite impulse response (FIR) model caucheteux2023evidence. D and d are feature dimensions before and after PCA. TR is the repetition time of fMRI.
  • Figure 3: Average brain score across different cognitive groups and activation layers (the bar plot), and the correlation between brain score and MoCA score (the blue line). The red dot means the correlation of that layer is significant.
  • Figure 4: (a) Averaged brain score map (activated by the 8th layer) for language-related ROIs. (b) Map of Pearson correlation coefficients between the averaged brain scores and MoCA scores. (c) Relationship between the averaged brain scores of ROIs and correlation coefficients. The number next to the point indicates the label number of the ROI in the Destrieux atlas. Red points represent ROIs with an adjusted $p$-value $< 0.05$; (d) Regression plot of MoCA scores and brain scores for the middle temporal gyrus (MTG).