Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Towards Zero-Shot Task-Generalizable Learning on fMRI

Jiyao Wang, Nicha C. Dvornek, Peiyu Duan, Lawrence H. Staib, James S. Duncan

TL;DR

The paper tackles the challenge of generalizing analyses across diverse task-based fMRI paradigms by introducing TA-GAT, a task-aware graph neural network. It combines a general-purpose GNN encoder with a memory bank of task representations and task-specific MLPs, using a composite loss including an orthogonal regularization term to encourage distinct task contexts. Empirically, TA-GAT improves performance on known tasks and demonstrates robust zero-shot generalization to unseen tasks on HCP data, with ablations confirming the benefit of the orthogonal loss. This approach enables more robust, context-aware brain pattern learning and suggests a path toward generalizable neuroimaging foundation models that leverage task structure.

Abstract

Functional MRI measuring BOLD signal is an increasingly important imaging modality in studying brain functions and neurological disorders. It can be acquired in either a resting-state or a task-based paradigm. Compared to resting-state fMRI, task-based fMRI is acquired while the subject is performing a specific task designed to enhance study-related brain activities. Consequently, it generally has more informative task-dependent signals. However, due to the variety of task designs, it is much more difficult than in resting state to aggregate task-based fMRI acquired in different tasks to train a generalizable model. To resolve this complication, we propose a supervised task-aware network TA-GAT that jointly learns a general-purpose encoder and task-specific contextual information. The encoder-generated embedding and the learned contextual information are then combined as input to multiple modules for performing downstream tasks. We believe that the proposed task-aware architecture can plug-and-play in any neural network architecture to incorporate the prior knowledge of fMRI tasks into capturing functional brain patterns.

Towards Zero-Shot Task-Generalizable Learning on fMRI

TL;DR

The paper tackles the challenge of generalizing analyses across diverse task-based fMRI paradigms by introducing TA-GAT, a task-aware graph neural network. It combines a general-purpose GNN encoder with a memory bank of task representations and task-specific MLPs, using a composite loss including an orthogonal regularization term to encourage distinct task contexts. Empirically, TA-GAT improves performance on known tasks and demonstrates robust zero-shot generalization to unseen tasks on HCP data, with ablations confirming the benefit of the orthogonal loss. This approach enables more robust, context-aware brain pattern learning and suggests a path toward generalizable neuroimaging foundation models that leverage task structure.

Abstract

Functional MRI measuring BOLD signal is an increasingly important imaging modality in studying brain functions and neurological disorders. It can be acquired in either a resting-state or a task-based paradigm. Compared to resting-state fMRI, task-based fMRI is acquired while the subject is performing a specific task designed to enhance study-related brain activities. Consequently, it generally has more informative task-dependent signals. However, due to the variety of task designs, it is much more difficult than in resting state to aggregate task-based fMRI acquired in different tasks to train a generalizable model. To resolve this complication, we propose a supervised task-aware network TA-GAT that jointly learns a general-purpose encoder and task-specific contextual information. The encoder-generated embedding and the learned contextual information are then combined as input to multiple modules for performing downstream tasks. We believe that the proposed task-aware architecture can plug-and-play in any neural network architecture to incorporate the prior knowledge of fMRI tasks into capturing functional brain patterns.

Paper Structure

This paper contains 20 sections, 5 equations, 1 figure, 3 tables.

Table of Contents

  1. Introduction
  2. Data
  3. Method
  4. Notation and Problem Definition
  5. Graph Construction
  6. Network Architecture
  7. General-purpose GNN Encoder
  8. fMRI Task Memory Bank
  9. Downstream MLP
  10. Loss Functions
  11. Experiments and Results
  12. Experimental Setup
  13. Main Results
  14. Ablation Study
  15. Discussion
  16. ...and 5 more sections

Figures (1)

  • Figure 1: Model architecture of TA-GAT. Blue and orange arrows denote the processing of two fMRI from distinct subjects and task stimuli. Task A to G corresponds to the 7 different fMRI tasks in HCP dataset.