Omni-fMRI: A Universal Atlas-Free fMRI Foundation Model

TL;DR

Omni-fMRI tackles the bottleneck of atlas-based fMRI representations by proposing a voxel-level, atlas-free foundation model that uses dynamic patching to manage the high dimensionality of 4D fMRI data. The method combines a dual-path, multi-scale embedding with a scale-aware masked autoencoding objective, enabling scalable pretraining across large, multi-dataset corpora while preserving fine-grained spatial structure. Empirically, Omni-fMRI achieves state-of-the-art transfer across diverse downstream tasks, including demographic prediction, image retrieval, emotion detection, and brain-state classification, and demonstrates strong linear-probing performance and data efficiency. The work provides a reproducible benchmarking framework across 11 datasets and delivers interpretable voxel-level representations that align with Neurosynth activations, suggesting robust, biologically meaningful modeling with atlas-free inputs.

Abstract

Self-supervised fMRI foundation models have shown promising transfer performance, yet most rely on predefined region-level parcellations that discard fine-grained voxel information and introduce atlas-dependent biases. We propose Omni-fMRI, an atlas-free foundation model that operates directly on voxel-level signals. To enable scalable pretraining on 49,497 fMRI sessions across nine datasets, Omni-fMRI introduces a dynamic patching mechanism that substantially reduces computational cost while preserving informative spatial structure. To support reproducibility and fair comparison, we establish a comprehensive benchmark suite spanning 11 datasets and a diverse set of resting-state and task-based fMRI tasks. Experimental results demonstrate that Omni-fMRI consistently outperforms existing foundation models, providing a scalable and reproducible framework for atlas-free brain representation learning. Code and logs are available.

Figures (6)

• Figure 1: Omni-fMRI reaches the best performance across a diverse array of resting-state and task-based fMRI benchmarks.
• Figure 2: Overview of the Omni-fMRI Pre-training Framework.(a) The proposed masked autoencoder architecture leverages a dynamic patching strategy to process 4D fMRI volumes. By filtering out background noise and adapting patch resolution, the model achieves computational efficiency while preserving fine-grained details. (b) Content-Adaptive Patch Allocation. A spatiotemporal complexity map is computed to guide tokenization: non-informative regions are pruned (Background Pruning), while foreground regions are dynamically assigned coarse (Red outline) or fine patch (Blue outline) based on local signal variability via a Complexity Gate. (c) Dual-Path Multi-Scale Embedding. To align heterogeneous patches into a unified latent space, we employ a dual-path projection module. Fine patches are projected directly, while coarser patches utilize a residual branch with Zero-initialized MLP (ZeroMLP) to fuse high-frequency structural details. (d) Scale-Aware Reconstruction. The reconstruction routes latent tokens to scale-specific prediction heads, utilizing distinct MLPs to reconstruct voxels at their appropriate resolution for high-fidelity restoration.
• Figure 3: 23-way HCP Task Accuracy under different few-shot levels. * denotes a large effect size with Cohen's d $\ge$ 0.8.
• Figure 4: Data Scaling and Linear Probing. Classification performance on the ADNI (AD) dataset across different models and settings. Note that pretraining on zero subjects corresponds to end-to-end supervised training. LP: linear probing.
• Figure 5: Visualization of key brain regions associated with Alzheimer.Left: Meta-analytic reference map from Neurosynth. Middle: Subject-averaged saliency map derived from Omni-fMRI using Integrated Gradients. Right: Corresponding saliency map from the NeuroSTORM baseline.