Unpaired Modality Translation for Pseudo Labeling of Histology Images

TL;DR

The paper tackles sparse annotations in histology image segmentation by introducing a microscopy pseudo labeling pipeline that leverages unpaired image translation between labeled ($L$) and unlabeled ($U$) domains. It combines SynDiff-based denoising diffusion GAN translations with two pseudo labeling strategies—tutorship ($L \rightarrow U$) and adaptation ($U \rightarrow L$)—to produce usable pseudo labels for $X_U$ and train/apply segmentation models. Across three domain-shift experiments (TEM↔TEM-MACAQUE, TEM↔SEM, TEM↔BF), tutoring performs best under moderate shifts (e.g., SEM) with a mean axon Dice of $0.736 \pm 0.005$ and myelin Dice of $0.652 \pm 0.005$, while adaptation can excel when shifts are larger (BF); pre-trained TEM models often fail on dissimilar targets. The approach demonstrates that pseudo labeling can substantially accelerate annotation by providing high-quality starting masks, with guidance on when to deploy adaptive versus tutoring paths across modality and domain differences.

Abstract

The segmentation of histological images is critical for various biomedical applications, yet the lack of annotated data presents a significant challenge. We propose a microscopy pseudo labeling pipeline utilizing unsupervised image translation to address this issue. Our method generates pseudo labels by translating between labeled and unlabeled domains without requiring prior annotation in the target domain. We evaluate two pseudo labeling strategies across three image domains increasingly dissimilar from the labeled data, demonstrating their effectiveness. Notably, our method achieves a mean Dice score of $0.736 \pm 0.005$ on a SEM dataset using the tutoring path, which involves training a segmentation model on synthetic data created by translating the labeled dataset (TEM) to the target modality (SEM). This approach aims to accelerate the annotation process by providing high-quality pseudo labels as a starting point for manual refinement.

Figures (4)

• Figure 1: Methodology overview. (a) Initial data and model (b) Unpaired translation between $L$ and $U$ via Adversarial Diffusion (c) Tutorship: Training a segmentation model on synthetic data $\{X_{L \rightarrow U}, Y_L\}$ for inference in $U$ (d) Adaptation: Applying a segmentation model pre-trained on $L$ to the translated images $X_{U \rightarrow L}$
• Figure 2: Overview of the two pseudo labeling strategies using TEM as the labeled dataset and SEM as the unlabeled data. (Top) The Tutorship strategy involves generating a synthetic dataset by translating labeled TEM images and training a proxy segmentation model on this synthetic data. (Bottom) The Adaptation strategy converts unlabeled SEM images to match the TEM data distribution, enabling segmentation with a pre-trained model.
• Figure 3: (Top) Image translation examples. (Bottom) Unsupervised translation reconstruction losses: Structural Similarity Index Measure (SSIM), Peak Signal-to-Noise Ratio (PSNR) and $L1$. The error bars represent the standard deviations among the validation images.
• Figure 4: Pseudo labeling results. (Top) Original and translated image alongside the pre-trained model prediction and pseudo labels. (Bottom) Dice scores under 5-fold cross-validation.