Unified Multi-Site Multi-Sequence Brain MRI Harmonization Enriched by Biomedical Semantic Style

TL;DR

This work tackles the challenge of non-biological site effects in large-scale multi-site brain MRI by proposing MMH, a two-stage diffusion-based harmonization framework. Stage I uses a voxel-space conditional diffusion model with gradient-anchored anatomical priors and sequence-specific EMA to produce unified sequence-domain representations across sites. Stage II fine-tunes globally harmonized MRIs to a target site using semantic style guidance from a Tri-Planar Attention CLIP (TPA-CLIP) module and a semantic style displacement loss, enabling robust, paired-data-free adaptation across multiple sequences. Extensive experiments on 4,163 T1w and T2w MRIs across multiple datasets show MMH achieves superior histogram alignment, preserves anatomical detail, improves segmentation and age-related biomarkers, and reduces site classification, highlighting its practicality for large-scale, multi-institutional neuroimaging analyses.

Abstract

Aggregating multi-site brain MRI data can enhance deep learning model training, but also introduces non-biological heterogeneity caused by site-specific variations (e.g., differences in scanner vendors, acquisition parameters, and imaging protocols) that can undermine generalizability. Recent retrospective MRI harmonization seeks to reduce such site effects by standardizing image style (e.g., intensity, contrast, noise patterns) while preserving anatomical content. However, existing methods often rely on limited paired traveling-subject data or fail to effectively disentangle style from anatomy. Furthermore, most current approaches address only single-sequence harmonization, restricting their use in real-world settings where multi-sequence MRI is routinely acquired. To this end, we introduce MMH, a unified framework for multi-site multi-sequence brain MRI harmonization that leverages biomedical semantic priors for sequence-aware style alignment. MMH operates in two stages: (1) a diffusion-based global harmonizer that maps MR images to a sequence-specific unified domain using style-agnostic gradient conditioning, and (2) a target-specific fine-tuner that adapts globally aligned images to desired target domains. A tri-planar attention BiomedCLIP encoder aggregates multi-view embeddings to characterize volumetric style information, allowing explicit disentanglement of image styles from anatomy without requiring paired data. Evaluations on 4,163 T1- and T2-weighted MRIs demonstrate MMH's superior harmonization over state-of-the-art methods in image feature clustering, voxel-level comparison, tissue segmentation, and downstream age and site classification.

Figures (10)

• Figure 1: Overview of the MMH framework for multi-site multi-sequence brain MRI harmonization through two-stage learning. (a) Sequence-Specific Global Harmonization: A Conditional Diffusion Model (CDM) is trained to harmonize multi-site, multi-sequence MRIs into unified sequence-specific domains. This stage employs style-agnostic gradient conditioning to anchor anatomy and sequence-specific EMA guidance to eliminate global intensity variations. (b) Target-Specific Fine Harmonization: The pre-trained CDM is fine-tuned to adapt the globally aligned MRIs to a specific target site's style, using a Tri-Planar Attention CLIP (TPA-CLIP) module. We introduce a semantic style displacement strategy, leveraging residuals between raw and globally aligned inputs, to guide robust adaptation without paired training data.
• Figure 2: Intensity histograms of (a) 16 T1w MRIs and (b) 16 T2w MRIs from the DWI-THP test set across 8 sites. The target domain (UCI) is highlighted in red.
• Figure 3: Harmonization results for (a) T1w and (b) T2w MRI from the DWI-THP test set. We display the Target (site UCI), the source MRI (Baseline, site DART), harmonized outputs from six SOTA methods, and our MMH, with absolute difference (Diff.) maps relative to the Target.
• Figure 4: Feature clustering results on (a) 40 unharmonized T1w MRIs; (b) 40 MMH-harmonized T1w MRIs; (c) 40 unharmonized T2w MRIs; and (d) 40 MMH-harmonized T2w MRIs, of 5 subjects across 8 sites in DWI-THP. Numbers denote the subjects, and colors represent the sites.
• Figure 5: Quantitative evaluation of voxel-level harmonization performance on the DWI-THP test set for (a) T1-weighted and (b) T2-weighted MRIs. Metrics (SSIM, PSNR, PCC, WD) compare each harmonized volume against the corresponding traveling subject reference at the target site (UCI). The violin plots illustrate the distribution of scores across the test cohort ($N=16$). Statistical significance between the proposed method (MMH) and competing baselines was assessed using the Wilcoxon signed-rank test ($*$: $p < 0.05$, $**$: $p < 0.01$, $***$: $p < 0.001$).