Mapping minds not averages: a scalable subject-specific manifold learning framework for neuroimaging data

TL;DR

This work introduces a scalable, subject-specific manifold learning framework for neuroimaging that captures spatial variability across individuals by learning a shared low-dimensional manifold with per-subject linear spatial maps. By employing a decomposed, parameter-efficient representation, the method supports voxelwise whole-brain data and large subject cohorts, while enabling generalization to unseen subjects. Through simulations and naturalistic fMRI datasets (Sherlock, Forrest Gump) and unstructured rs-fMRI in schizophrenia, the approach yields superior reconstruction and classification performance and reveals clinically relevant, subject-specific brain patterns. The results suggest a practical pathway for individualized brain activity modeling with potential applications in computational neuroscience and clinical research, including scalable analysis and foundation-model integration.

Abstract

Mental and cognitive representations are believed to reside on low-dimensional, non-linear manifolds embedded within high-dimensional brain activity. Uncovering these manifolds is key to understanding individual differences in brain function, yet most existing machine learning methods either rely on population-level spatial alignment or assume data that is temporally structured, either because data is aligned among subjects or because event timings are known. We introduce a manifold learning framework that can capture subject-specific spatial variations across both structured and temporally unstructured neuroimaging data. On simulated data and two naturalistic fMRI datasets (Sherlock and Forrest Gump), our framework outperforms group-based baselines by recovering more accurate and individualized representations. We further show that the framework scales efficiently to large datasets and generalizes well to new subjects. To test this, we apply the framework to temporally unstructured resting-state fMRI data from individuals with schizophrenia and healthy controls. We further apply our method to a large resting-state fMRI dataset comprising individuals with schizophrenia and controls. In this setting, we demonstrate that the framework scales efficiently to large populations and generalizes robustly to unseen subjects. The learned subject-specific spatial maps our model finds reveal clinically relevant patterns, including increased activation in the basal ganglia, visual, auditory, and somatosensory regions, and decreased activation in the insula, inferior frontal gyrus, and angular gyrus. These findings suggest that our framework can uncover clinically relevant subject-specific brain activity patterns. Our approach thus provides a scalable and individualized framework for modeling brain activity, with applications in computational neuroscience and clinical research.

Figures (6)

• Figure 1: Subfigure a: The three main models tested in our framework, subject-specific parameters have different colors across subjects, and parameters shared across subjects are gray. The Group model assumes the same spatial weights and neural network for all subjects in a dataset, and requires the smallest number of parameters. This is currently the most common way neural networks are applied to neuroimaging data. The Subject model assumes a different spatial weights matrix for each subject, but shares the same neural network across the subjects in the dataset. This model requires the largest number of parameters, especially with many subjects or large spatial inputs. The Decomposed model assumes a matrix decomposition, and assumes that only the singular values are different for each subject. This greatly reduces the number of parameters used compared with the Subject model, and allows it to both capture subject-specific spatial maps and scale to a large number of subjects or large spatial maps. Subfigure b: The simulated data experiment. Each simulated subject is a randomly rotated version of the half moon dataset. We train a multi-layer perceptron (MLP) using the three different models, and show that the Group model can not capture the subject differences, even when a neural network is used. The Subject and Decomposed models achieve almost perfect performance, and the singular values of the Decomposed model projected onto the first two principal components form a circle. This circle corresponds to the different subjects, and clearly captures the rotational generation process (the unit circle).
• Figure 2: The classification results in this figure are calculated with a radial basis function (RBF) support vector machine (SVM) for auto-encoder models that have embedded an unseen 50% of the timeseries. The top row shows performance for the auditory, early visual, posteromedial cortex regions of interest (ROIs), and whole-brain data from the Sherlock dataset chen2017shared. Separate RBF-SVMs are trained for the music presence (binary) and indoor vs outdoor (binary) classification. The bottom row shows the performance for the auditory, precuneus, and cognitive control ROIs, using whole-brain data from the auditory Forrest Gump dataset hanke2014high. The three classification types are flow-of-time (4-way), whether a scene is interior or exterior (binary), and time-of-day (binary). In almost all cases, the Subject and Decomposed models significantly outperform the Group and manifold-regularized multiple decoder, autoencoder (MRMD-AE) models. The results for the MRMD-AE are taken from the original paper huang2022learning. Especially for whole-brain data, classification performance is high and significantly better, but the Subject model can not be evaluated in this setting due to bad memory scaling.
• Figure 3: Reconstruction improvements in terms of percentage reduction in mean squared error over the Group model on the test set. For both datasets and each region of interest (ROI) or whole-brain data, both the Subject and Decomposed models significantly outperform the Group model in terms of generalization to new fMRI data from the same subjects. This indicates that using subject-specific spatial maps helps the autoencoder capture more of the variance in the data in a generalizable manner.
• Figure 4: Subfigure a: We evaluate reconstruction performance as a function of the percentage of data we fine-tune the subject-specific weights of unseen subjects on. The total number of timesteps is 157, and even with 1% of those timesteps, our model outperforms the group layer in terms of reconstruction. During fine-tuning all layers in the model are frozen except for the subject-specific weights for unseen subjects. Subfigure b: Group distribution difference plots of subject-specific weights in a 2D PCA space. Individuals with schizophrenia are more concentrated in the left side of the plot, whereas control subjects are more concentrated towards the right side in the plot. These concentrations are also replicated for fine-tuned weights from unseen subjects. Subfigure c: A visualization of the subject-specific weight updates for unseen subjects (on the fBIRN rs-fMRI schizophrenia dataset) during fine-tuning. Over time, many of the unseen subject weights move to areas that contain higher densities of training subjects with that same diagnosis.
• Figure 5: Significantly different brain activations between schizophrenia patients and control subjects as indicated by our Decomposed model. These brain regions are a selection from all the significant regions. We generally find significantly increased basal ganglia, visual, auditory, and somatosensory activations for schizophrenia patients. The inferior frontal gyrus, angular gyrus, and insula show significantly decreased activity however. The significance levels are corrected using the false discovery rate based on $64$ tests.