Feedback-Driven Pseudo-Label Reliability Assessment: Redefining Thresholding for Semi-Supervised Semantic Segmentation

TL;DR

The paper addresses the challenge of threshold-based pseudo-label filtering in semi-supervised semantic segmentation under data scarcity. It introduces ENCORE, a dynamic thresholding framework combining Class-Aware Confidence Calibration (CAC) and Adaptive Confidence Thresholding (ACT) to adaptively select pseudo-labels based on class-wise reliability and training feedback. CAC estimates per-class true-positive confidence from labeled data to initialize class-specific filters, while ACT uses multiple assessor networks to iteratively optimize thresholds via Dice score on labeled batches, updating thresholds as training progresses. Empirical results across five medical imaging datasets show that ENCORE consistently boosts performance across multiple SSL frameworks and architectures, particularly in extreme low-data regimes, and ablations confirm the complementary benefits of CAC and ACT. The work reduces manual hyperparameter tuning in pseudo-supervision and offers a practical, plug-in enhancement for robust semi-supervised segmentation.

Abstract

Semi-supervised learning leverages unlabeled data to enhance model performance, addressing the limitations of fully supervised approaches. Among its strategies, pseudo-supervision has proven highly effective, typically relying on one or multiple teacher networks to refine pseudo-labels before training a student network. A common practice in pseudo-supervision is filtering pseudo-labels based on pre-defined confidence thresholds or entropy. However, selecting optimal thresholds requires large labeled datasets, which are often scarce in real-world semi-supervised scenarios. To overcome this challenge, we propose Ensemble-of-Confidence Reinforcement (ENCORE), a dynamic feedback-driven thresholding strategy for pseudo-label selection. Instead of relying on static confidence thresholds, ENCORE estimates class-wise true-positive confidence within the unlabeled dataset and continuously adjusts thresholds based on the model's response to different levels of pseudo-label filtering. This feedback-driven mechanism ensures the retention of informative pseudo-labels while filtering unreliable ones, enhancing model training without manual threshold tuning. Our method seamlessly integrates into existing pseudo-supervision frameworks and significantly improves segmentation performance, particularly in data-scarce conditions. Extensive experiments demonstrate that integrating ENCORE with existing pseudo-supervision frameworks enhances performance across multiple datasets and network architectures, validating its effectiveness in semi-supervised learning.

Figures (5)

• Figure 1: Comparison of Dice scores across different confidence thresholds for various labeled data percentages on the ACDC and LA datasets with AD-MT AD-MT. The plots illustrate how segmentation performance fluctuates with threshold selection, particularly in low-data regimes. As the labeled data fraction decreases, model sensitivity to threshold choice increases, making optimal threshold selection more challenging. Notably, a fixed global threshold often fails to generalize well across different dataset conditions, underscoring the need for an adaptive thresholding strategy.
• Figure 2: Overview of ENCORE: A feedback-driven pseudo-label refinement framework integrating Class-Aware Confidence Calibration (CAC) and Adaptive Confidence Thresholding (ACT) to enhance pseudo-label reliability and improve semi-supervised segmentation.
• Figure 3: Comparison between CAC-based confidence thresholds and fixed thresholds for Cataract-1K dataset (1/8 labeled data).
• Figure 4: Qualitative comparisons of state-of-the-art methods with and without ENCORE on the Cataract-1K, LA, and ACDC datasets.
• Figure 5: Kernel density estimates from various methods trained on 1 labeled ACDC dataset. From top to bottom, the plots display features corresponding to the right ventricle, myocardium, and left ventricle classes.