Confidence-Weighted Semi-Supervised Learning for Skin Lesion Segmentation Using Hybrid CNN-Transformer Networks

TL;DR

The paper tackles automated skin lesion segmentation under scarce annotations by integrating a MIM-pretrained, uncertainty-aware teacher with a semi-supervised learner. A hybrid CNN–Transformer student, equipped with cross-attention skip connections, benefits from confidence-weighted pseudo-labels and a joint objective that n incorporates supervised Dice+BCE, consistency regularization, and entropy control. Empirical results on ISIC-2016 and PH2 demonstrate strong gains in Dice and IoU in low-label regimes and good cross-dataset generalization, with ablations validating the importance of uncertainty filtering, MIM pretraining, and the hybrid backbone. The approach offers a practical SSL framework for dermoscopic analysis, with potential impact on clinical workflows and avenues for further efficiency and robustness improvements.

Abstract

Automated skin lesion segmentation through dermoscopic analysis is essential for early skin cancer detection, yet remains challenging due to limited annotated training data. We present MIRA-U, a semi-supervised framework that combines uncertainty-aware teacher-student pseudo-labeling with a hybrid CNN-Transformer architecture. Our approach employs a teacher network pre-trained via masked image modeling to generate confidence-weighted soft pseudo-labels, which guide a U-shaped CNN-Transformer student network featuring cross-attention skip connections. This design enhances pseudo-label quality and boundary delineation, surpassing reconstruction-based and CNN-only baselines, particularly in low-annotation regimes. Extensive evaluation on ISIC-2016 and PH2 datasets demonstrates superior performance, achieving a Dice Similarity Coefficient (DSC) of 0.9153 and Intersection over Union (IoU) of 0.8552 using only 50% labeled data. Code is publicly available on GitHub.

Figures (4)

• Figure 1: Overview of the proposed MIRA-U framework for semi-supervised skin lesion segmentation. The framework integrates two main components: (i) a teacher network trained with MIM to learn context-rich representations, which is further used for uncertainty-aware pseudo-label generation via Monte Carlo dropout, and (ii) a student network based on a hybrid CNN–transformer segmentation backbone. The teacher generates confidence-weighted pseudo-labels from unlabeled images under weak augmentation, whereas the student is trained jointly on labeled data and high-confidence pseudo-labeled data under strong augmentation. Cross-attention within the decoder fuses the encoder features to refine the segmentation outputs. The teacher is updated as the exponential moving average (EMA) of the student, which enables stable training and effective knowledge transfer.
• Figure 2: Examples of pseudo-label generation in MIRA-U. The first row shows the input images, the second row shows the ground-truth masks, and the third row shows the pseudo-labels from the teacher network. Although these pseudo-labels roughly capture the lesion regions, they contain noise and boundary artifacts that are reduced through uncertainty-aware filtering before training the student.
• Figure 3: Examples of MIRA-U results on ISIC-2016. The first column shows input images, the second column ground-truth masks, and the third column predictions from MIRA-U.
• Figure 4: Examples of MIRA-U results on Ph2. The first column shows input images, the second column ground-truth masks, and the third column predictions from MIRA-U.