EAUWSeg: Eliminating annotation uncertainty in weakly-supervised medical image segmentation

TL;DR

This work addresses annotation uncertainty in weakly-supervised medical image segmentation by introducing bounded polygon annotations (BPAnno) that use two polygons per lesion. The authors propose EAUWSeg, a learning framework comprising a BPAnno-guided segmentation backbone plus a Classification-Guided Confidence Generator and a Confidence-Auxiliary Consistency Learner, enabling reliable supervision in uncertain regions via entropy-based confidence and pixel-wise contrastive learning. Empirical results on ISIC2017 and Kvasir-SEG show that EAUWSeg achieves state-of-the-art performance among weakly-supervised methods and often surpasses fully-supervised baselines while reducing annotation workload to under 20% of dense labeling; the approach also generalizes across backbones and to ISIC2018. Overall, BPAnno+EAUWSeg offers a cost-effective, stable, and robust pathway for high-performance medical image segmentation under weak supervision with explicit uncertainty handling.

Abstract

Weakly-supervised medical image segmentation is gaining traction as it requires only rough annotations rather than accurate pixel-to-pixel labels, thereby reducing the workload for specialists. Although some progress has been made, there is still a considerable performance gap between the label-efficient methods and fully-supervised one, which can be attributed to the uncertainty nature of these weak labels. To address this issue, we propose a novel weak annotation method coupled with its learning framework EAUWSeg to eliminate the annotation uncertainty. Specifically, we first propose the Bounded Polygon Annotation (BPAnno) by simply labeling two polygons for a lesion. Then, the tailored learning mechanism that explicitly treat bounded polygons as two separated annotations is proposed to learn invariant feature by providing adversarial supervision signal for model training. Subsequently, a confidence-auxiliary consistency learner incorporates with a classification-guided confidence generator is designed to provide reliable supervision signal for pixels in uncertain region by leveraging the feature presentation consistency across pixels within the same category as well as class-specific information encapsulated in bounded polygons annotation. Experimental results demonstrate that EAUWSeg outperforms existing weakly-supervised segmentation methods. Furthermore, compared to fully-supervised counterparts, the proposed method not only delivers superior performance but also costs much less annotation workload. This underscores the superiority and effectiveness of our approach.

Figures (6)

• Figure 1: Comparison of the typical weak annotation methods and our proposed bounded polygon annotations, including the annotation strategies and the annotation uncertainty. The yellow curves show groundtruth segmentation. The black and gray denote the certain and uncertain regions, respectively.
• Figure 2: The overall framework of our proposed EAUWSeg. It includes a segmentation model supervised by two bounded polygons and a multi-class classification labels. Additionally, a confidence-auxiliary consistency learner is integrated to focus on compact feature learning in the uncertain region. During training, the extracted feature $f_{\mathcal{S}}(\cdot)$ is input into the segmentation head to generate feature representation for lesions. Simultaneously, the classification head supervised by $y^{c}$ is used to generate the confidence of uncertain pixels, the embedding head guided by bounded polygon annotations and confidence of uncertain pixels is utilized to construct a compact feature space.
• Figure 3: Qualitative comparison of different methods on ISIC2017 (top three rows) and Kvasir-SEG (bottom three rows). The green and blue contours indicate the prediction and groundtruth, respectively.
• Figure 4: Performance comparison of EAUWSeg combined with different backbones on the ISIC2017 test set.
• Figure 5: Error analysis on the ISIC2017 test set. Both inside and outside a band of specific width are illustrated.