MNM : Multi-level Neuroimaging Meta-analysis with Hyperbolic Brain-Text Representations

TL;DR

This work tackles the small-sample problem in neuroimaging meta-analysis by aligning brain activation maps and textual descriptions in a shared hyperbolic space. It introduces MNM, which embeds brains and texts within the Lorentz model $\mathbb{L}^n$ to exploit exponential growth and hierarchical structure, using an angle-based contrastive term, centroid distance regularization, and a brain hierarchy penalty for joint optimization of semantic alignment and hierarchy. Key contributions include a hyperbolic brain-text representation for robust cross-modal mapping, explicit brain structural hierarchy guidance, and improved performance on cross-modal retrieval and activation-map prediction over Euclidean baselines. The approach demonstrates robustness across parcellations (e.g., Yeo7, Yeo17) and enhances interpretability, enabling scalable multi-level meta-analysis of neuroimaging literature.

Abstract

Various neuroimaging studies suffer from small sample size problem which often limit their reliability. Meta-analysis addresses this challenge by aggregating findings from different studies to identify consistent patterns of brain activity. However, traditional approaches based on keyword retrieval or linear mappings often overlook the rich hierarchical structure in the brain. In this work, we propose a novel framework that leverages hyperbolic geometry to bridge the gap between neuroscience literature and brain activation maps. By embedding text from research articles and corresponding brain images into a shared hyperbolic space via the Lorentz model, our method captures both semantic similarity and hierarchical organization inherent in neuroimaging data. In the hyperbolic space, our method performs multi-level neuroimaging meta-analysis (MNM) by 1) aligning brain and text embeddings for semantic correspondence, 2) guiding hierarchy between text and brain activations, and 3) preserving the hierarchical relationships within brain activation patterns. Experimental results demonstrate that our model outperforms baselines, offering a robust and interpretable paradigm of multi-level neuroimaging meta-analysis via hyperbolic brain-text representation.

Figures (4)

• Figure 1: Motivation of MNM. (a) The brain exhibits a spatial hierarchy, where broad regions can be further subdivided into more specific regions. (b) Each brain region can correspond to various articles covering diverse topics, such as functionalities.
• Figure 2: Concept of Our Objective Functions. (a) The angle-based contrastive loss minimizes the exterior angle between a text embedding and its corresponding brain activation embedding. (b) The centroid loss encourages the centroid of brain embeddings $\mathbf{C}_\mathrm{brain}$ to be positioned closer to the hyperboloid origin than the centroid of text embeddings $\mathbf{C}_\mathrm{text}$. (c) The hierarchical loss aligns brain embeddings along the hyperbolic time axis by the number of activated regions for brain structural hierarchy.
• Figure 3: Effect of Brain Hierarchical Loss. (a) Histogram between $x_\mathrm{time}$ and the number of activated ROIs without $\mathcal{L}_\mathrm{hier}$, showing $-0.30$ of Kendall's $\tau$kendalltau (b) Same histogram with $\mathcal{L}_\mathrm{hier}$, decreasing Kendall's $\tau$ to $-0.58$. (c) 2D Poincaré disk projection of Yeo7 and Yeo17 network yeo embeddings (left: w/o $\mathcal{L}_\mathrm{hier}$, right: w/ $\mathcal{L}_\mathrm{hier}$).
• Figure 4: Reconstructed Brain Activation Map from a Neuroscience Article visuospatial. While NeuroConText neurocontext2 fails to capture activation in the intraparietal sulcus using Yeo17 networks, MNM assigns high similarity scores to them as in the ground truth.