BrainMass: Advancing Brain Network Analysis for Diagnosis with Large-scale Self-Supervised Learning

TL;DR

The BrainMass framework for brain network self-supervised learning via mask modeling and feature alignment is proposed, and Latent Representation Alignment module is utilized to regularize augmented brain networks of the same participant with similar topological properties to yield similar latent representations by aligning their latent embeddings.

Abstract

Foundation models pretrained on large-scale datasets via self-supervised learning demonstrate exceptional versatility across various tasks. Due to the heterogeneity and hard-to-collect medical data, this approach is especially beneficial for medical image analysis and neuroscience research, as it streamlines broad downstream tasks without the need for numerous costly annotations. However, there has been limited investigation into brain network foundation models, limiting their adaptability and generalizability for broad neuroscience studies. In this study, we aim to bridge this gap. In particular, (1) we curated a comprehensive dataset by collating images from 30 datasets, which comprises 70,781 samples of 46,686 participants. Moreover, we introduce pseudo-functional connectivity (pFC) to further generates millions of augmented brain networks by randomly dropping certain timepoints of the BOLD signal. (2) We propose the BrainMass framework for brain network self-supervised learning via mask modeling and feature alignment. BrainMass employs Mask-ROI Modeling (MRM) to bolster intra-network dependencies and regional specificity. Furthermore, Latent Representation Alignment (LRA) module is utilized to regularize augmented brain networks of the same participant with similar topological properties to yield similar latent representations by aligning their latent embeddings. Extensive experiments on eight internal tasks and seven external brain disorder diagnosis tasks show BrainMass's superior performance, highlighting its significant generalizability and adaptability. Nonetheless, BrainMass demonstrates powerful few/zero-shot learning abilities and exhibits meaningful interpretation to various diseases, showcasing its potential use for clinical applications.

Figures (6)

• Figure 1: Illustration of (i) the construction of pFC, (ii) the training phase of BrainMass method, including an MRM (an MRM network) and an LRA (an online network and a target network) module, and (iii) the inference phase of BrainMass.
• Figure 2: The effect on the dropping rate on eight internal tasks.
• Figure 3: The effect on the model size.
• Figure 4: The accuracy performances on seven external tasks.
• Figure 5: Heatmaps of the Transformer encoder attention maps on 7 tasks, including the averaged attention maps (the first row), those of the first layer (the second row), and the last layer (the third row). The values in heatmaps are normalized into 0 to 1.