USE: Uncertainty Structure Estimation for Robust Semi-Supervised Learning

TL;DR

It can be concluded that the proposed approach reframes unlabeled data quality control as a structural assessment problem, and considers it as a necessary component for reliable and efficient SSL in realistic mixed-distribution environments.

Abstract

In this study, a novel idea, Uncertainty Structure Estimation (USE), a lightweight, algorithm-agnostic procedure that emphasizes the often-overlooked role of unlabeled data quality is introduced for Semi-supervised learning (SSL). SSL has achieved impressive progress, but its reliability in deployment is limited by the quality of the unlabeled pool. In practice, unlabeled data are almost always contaminated by out-of-distribution (OOD) samples, where both near-OOD and far-OOD can negatively affect performance in different ways. We argue that the bottleneck does not lie in algorithmic design, but rather in the absence of principled mechanisms to assess and curate the quality of unlabeled data. The proposed USE trains a proxy model on the labeled set to compute entropy scores for unlabeled samples, and then derives a threshold, via statistical comparison against a reference distribution, that separates informative (structured) from uninformative (structureless) samples. This enables assessment as a preprocessing step, removing uninformative or harmful unlabeled data before SSL training begins. Through extensive experiments on imaging (CIFAR-100) and NLP (Yelp Review) data, it is evident that USE consistently improves accuracy and robustness under varying levels of OOD contamination. Thus, it can be concluded that the proposed approach reframes unlabeled data quality control as a structural assessment problem, and considers it as a necessary component for reliable and efficient SSL in realistic mixed-distribution environments.

Figures (5)

• Figure 1: This figure displays the entropy density distribution computed using a proxy model trained on 200 labeled samples from the CIFAR-100 dataset. The blue curve represents in-distribution (ID) samples (CIFAR-100), concentrated in the low-entropy region; the orange curve represents near-OOD samples (Tiny ImageNet), exhibiting an approximately uniform distribution; and the green curve represents far-OOD samples (SVHN), concentrated in the high-entropy region.
• Figure 2: Illustration of the USE threshold. The threshold $u^\ast$ corresponds to the first descending downward crossing of the entropy density, separating informative low-entropy samples from uninformative high-entropy ones. The intersection with the reference curve density $F_0'(u)$ can be interpreted as the point of maximum structural discrepancy.
• Figure 3: Overview of the USE pipeline. A proxy model trained on labeled data computes entropy scores for the unlabeled pool. By comparing the empirical entropy distribution with a structureless reference curve, a threshold $u^\ast$ is determined to separate structured from structureless samples, ensuring that downstream SSL is trained only on data with meaningful structure.
• Figure 4: Performance comparison of SSL methods on CIFAR-100 with 200 labeled samples. For each method, the six bars show the top-1 accuracy difference (USE – baseline) under contamination ratios $r \in \{0.0, 0.2, 0.4, 0.5, 0.6, 0.8\}$, with Tiny ImageNet (near-OOD) and SVHN (far-OOD) used as contamination sources. Green bars indicate that USE outperforms the baseline, while red bars indicate the opposite.
• Figure 5: Performance comparison of SSL methods on CIFAR-100 with 1000 labeled samples. For each method, the six bars show the top-1 accuracy difference (USE - baseline) under contamination ratios $r \in \{0.0, 0.2, 0.4, 0.5, 0.6, 0.8\}$, with Tiny ImageNet (near-OOD) and SVHN (far-OOD) used as contamination sources. Green bars indicate that USE outperforms the baseline, while red bars indicate the opposite.