TransMedSeg: A Transferable Semantic Framework for Semi-Supervised Medical Image Segmentation

TL;DR

TransMedSeg tackles domain shift and reliance on labeled data in semi-supervised medical image segmentation by introducing Transferable Semantic Augmentation (TSA) within a teacher–student framework, augmented by memory-augmented feature statistics. By explicitly modeling cross-domain and intra-domain semantic relationships and employing an implicit augmentation strategy with an upper-bound loss, the method achieves robust cross-domain transfer without explicit data generation. Empirical results on multiple benchmarks show state-of-the-art performance under low-label regimes and across imaging modalities, with ablations confirming the importance of TSA. The approach advances transferable representation learning in medical imaging and offers a plug-in mechanism to improve other SSMIS methods.

Abstract

Semi-supervised learning (SSL) has achieved significant progress in medical image segmentation (SSMIS) through effective utilization of limited labeled data. While current SSL methods for medical images predominantly rely on consistency regularization and pseudo-labeling, they often overlook transferable semantic relationships across different clinical domains and imaging modalities. To address this, we propose TransMedSeg, a novel transferable semantic framework for semi-supervised medical image segmentation. Our approach introduces a Transferable Semantic Augmentation (TSA) module, which implicitly enhances feature representations by aligning domain-invariant semantics through cross-domain distribution matching and intra-domain structural preservation. Specifically, TransMedSeg constructs a unified feature space where teacher network features are adaptively augmented towards student network semantics via a lightweight memory module, enabling implicit semantic transformation without explicit data generation. Interestingly, this augmentation is implicitly realized through an expected transferable cross-entropy loss computed over the augmented teacher distribution. An upper bound of the expected loss is theoretically derived and minimized during training, incurring negligible computational overhead. Extensive experiments on medical image datasets demonstrate that TransMedSeg outperforms existing semi-supervised methods, establishing a new direction for transferable representation learning in medical image analysis.

Figures (5)

• Figure 1: Overview of TransMedSeg. TransMedSeg enhances the semantic alignment of source features, facilitating the successful adaptation of the final classifier from the student network to the teacher network.
• Figure 2: Illustration of TransMedSeg. For each class, we leverage the inter-domain feature mean difference (indicated by the orange dashed arrow) and the target intra-class covariance to enhance teacher features, aligning them with the student's style.
• Figure 3: (a–b) T-SNE visualization of feature embeddings w/o and with $\mathcal{L}_{\text{tsa}}$. (c-d) Sensitivity analysis of $\mathcal{L}_{\text{tsa}}$ with respect to the weighting coefficient $\beta$ on ACDC dataset.
• Figure 4: Qualitative visualization results from the ACDC dataset.
• Figure 5: Qualitative visualization results from the Pancreas-NIH dataset.