MAPSeg: Unified Unsupervised Domain Adaptation for Heterogeneous Medical Image Segmentation Based on 3D Masked Autoencoding and Pseudo-Labeling

TL;DR

MAPSeg tackles domain shift in 3D medical image segmentation by unifying three strategies: 3D masked autoencoding for self-supervised pretraining, 3D masked pseudo-labeling for domain-adaptive self-training, and a global-local collaboration mechanism to fuse context at multiple scales. It extends naturally from centralized to federated and test-time unsupervised domain adaptation, enabling robust 3D segmentation across cross-sequence, cross-site, cross-age, and cross-modality scenarios without target labels for validation. The framework demonstrates large Dice gains over state-of-the-art methods on private infant brain MRI and public cardiac CT–MRI tasks, underscoring practical value for multi-center studies while preserving data privacy. Overall, MAPSeg provides a versatile, scalable approach for real-world heterogeneous medical image segmentation with strong cross-domain performance and broad applicability.

Abstract

Robust segmentation is critical for deriving quantitative measures from large-scale, multi-center, and longitudinal medical scans. Manually annotating medical scans, however, is expensive and labor-intensive and may not always be available in every domain. Unsupervised domain adaptation (UDA) is a well-studied technique that alleviates this label-scarcity problem by leveraging available labels from another domain. In this study, we introduce Masked Autoencoding and Pseudo-Labeling Segmentation (MAPSeg), a $\textbf{unified}$ UDA framework with great versatility and superior performance for heterogeneous and volumetric medical image segmentation. To the best of our knowledge, this is the first study that systematically reviews and develops a framework to tackle four different domain shifts in medical image segmentation. More importantly, MAPSeg is the first framework that can be applied to $\textbf{centralized}$, $\textbf{federated}$, and $\textbf{test-time}$ UDA while maintaining comparable performance. We compare MAPSeg with previous state-of-the-art methods on a private infant brain MRI dataset and a public cardiac CT-MRI dataset, and MAPSeg outperforms others by a large margin (10.5 Dice improvement on the private MRI dataset and 5.7 on the public CT-MRI dataset). MAPSeg poses great practical value and can be applied to real-world problems. GitHub: https://github.com/XuzheZ/MAPSeg/.

Figures (7)

• Figure 1: (a). Illustrations of four different domain shifts in medical images. (b). Overview of different UDA settings and how MAPSeg can fit into different scenarios.
• Figure 1: Illustrations of 3D ResNet Block and 3D Atrous Spatial Pyramid Pooling (ASPP) layer.
• Figure 2: Components of the proposed MAPSeg framework. (a) 3D multi-scale masked autoencoding. (b) 3D masked pseudo labeling in source and target domains. (c) 3D Global-local collaboration.
• Figure 2: Downstream cross-sequence centralized UDA performance vs. MAE pretraining iterations.
• Figure 3: Ablation studies on masking ratio, patch size, and pretrain data. Experiments on masking ratio and patch size are conducted on cross-sequence task.