SRA-Seg: Synthetic to Real Alignment for Semi-Supervised Medical Image Segmentation

TL;DR

This work tackles the synthetic-to-real domain gap in semi-supervised medical image segmentation by introducing SRA-Seg, a framework that explicitly aligns synthetic and real distributions via a Similarity-Alignment loss built on frozen DINO-v2 embeddings, along with soft edge blending and EMA-based pseudo-labeling. It combines Soft-Segmentation Loss on soft targets with a SA-Loss that minimizes distances between synthetic and real feature embeddings, and employs soft-mix augmentation to create smooth boundary transitions. Evaluations on ACDC and FIVES show that, with only $10\%$ real labeled data and $90\%$ synthetic unlabeled data, SRA-Seg outperforms existing SSL methods using synthetic data and matches the performance of methods trained with real unlabeled data, achieving Dice scores of $89.34$ on ACDC and $84.42$ on FIVES. The results underscore the potential of principled synthetic-real feature alignment to alleviate annotation bottlenecks in medical imaging, while ablations and FID analyses highlight the importance of data quality and all proposed components.

Abstract

Synthetic data, an appealing alternative to extensive expert-annotated data for medical image segmentation, consistently fails to improve segmentation performance despite its visual realism. The reason being that synthetic and real medical images exist in different semantic feature spaces, creating a domain gap that current semi-supervised learning methods cannot bridge. We propose SRA-Seg, a framework explicitly designed to align synthetic and real feature distributions for medical image segmentation. SRA-Seg introduces a similarity-alignment (SA) loss using frozen DINOv2 embeddings to pull synthetic representations toward their nearest real counterparts in semantic space. We employ soft edge blending to create smooth anatomical transitions and continuous labels, eliminating the hard boundaries from traditional copy-paste augmentation. The framework generates pseudo-labels for synthetic images via an EMA teacher model and applies soft-segmentation losses that respect uncertainty in mixed regions. Our experiments demonstrate strong results: using only 10% labeled real data and 90% synthetic unlabeled data, SRA-Seg achieves 89.34% Dice on ACDC and 84.42% on FIVES, significantly outperforming existing semi-supervised methods and matching the performance of methods using real unlabeled data.

Figures (8)

• Figure 1: Kernel Density Estimation (KDE) graphs showcasing domain mismatch between labeled (green line) and unlabeled (blue line) data for "Right Ventricle" class of the ACDC dataset acdc. (a) UNetunet with Real Unlabeled Data (b) UNetunet with Synthetic Unlabeled Data (c) BCPbai2023bidirectionalcopypastesemisupervisedmedical with Synthetic Unlabeled Data (d) Proposed SRA-Seg with Synthetic Unlabeled Data.
• Figure 2: Overview of the proposed SRA-Seg method
• Figure 3: Synthetic Data Generated by the StyleGAN2-ADAkarras2020traininggenerativeadversarialnetworks for the ACDC datasetacdc (top) and the FIVES datasetJin2022 (bottom). Left to right - the first 3 are the original images, the next 3 are the images generated using 5% real data during training, and the last 3 images are generated using 10% real data.
• Figure 4: Soft Edge Blending component of SRA-Seg. The labeled and unlabeled images as well as the corresponding segmentation masks are mixed through a cropping and soft mask blending to reduce the sharp edges. A zoomed-in comparison of blending results between BCPbai2023bidirectionalcopypastesemisupervisedmedical and SRA-Seg is shown.
• Figure 5: Comparison of real‐image usage (bar height) and resulting Dice scores (red markers) for BCPbai2023bidirectionalcopypastesemisupervisedmedical versus SRA-Seg on the ACDCacdc and FIVESJin2022 datasets.