AllMatch: Exploiting All Unlabeled Data for Semi-Supervised Learning

TL;DR

AllMatch tackles the underutilization of unlabeled data in semi-supervised learning by introducing two mechanisms: class-specific adaptive thresholds (CAT) that couple global learning status with per-class weight norms, and binary classification consistency (BCC) that leverages top-k candidate classes to supervise all unlabeled samples. CAT provides a global threshold and class-wise adjustments, enabling more accurate pseudo-label selection for hard classes, while BCC ensures consistent candidate–negative divisions across augmented views. Together, these strategies yield improved pseudo-label quality and full utilization of the unlabeled set, achieving state-of-the-art results on multiple balanced and imbalanced benchmarks. The approach demonstrates strong practical impact by robustly exploiting unlabeled data in diverse data regimes and architectures.

Abstract

Existing semi-supervised learning algorithms adopt pseudo-labeling and consistency regulation techniques to introduce supervision signals for unlabeled samples. To overcome the inherent limitation of threshold-based pseudo-labeling, prior studies have attempted to align the confidence threshold with the evolving learning status of the model, which is estimated through the predictions made on the unlabeled data. In this paper, we further reveal that classifier weights can reflect the differentiated learning status across categories and consequently propose a class-specific adaptive threshold mechanism. Additionally, considering that even the optimal threshold scheme cannot resolve the problem of discarding unlabeled samples, a binary classification consistency regulation approach is designed to distinguish candidate classes from negative options for all unlabeled samples. By combining the above strategies, we present a novel SSL algorithm named AllMatch, which achieves improved pseudo-label accuracy and a 100% utilization ratio for the unlabeled data. We extensively evaluate our approach on multiple benchmarks, encompassing both balanced and imbalanced settings. The results demonstrate that AllMatch consistently outperforms existing state-of-the-art methods.

Figures (9)

• Figure 1: Pilot study for pseudo-label quality on CIFAR-10 with 40 labels. Regarding SoftMatch, pseudo-labels with a confidence lower than $\mu_t-\sigma_t$ are assigned close-to-zero weights and thus considered dropped in the analysis. Here, $\mu_t / \sigma_t$ denotes the estimated mean/std of the overall confidence on unlabeled data, respectively.
• Figure 2: Pipeline of the class-specific adaptive threshold (CAT) mechanism. CAT employs the average confidence on unlabeled data as the global threshold and subsequently utilizes the classifier weights to establish class-specific thresholds.
• Figure 3: Pipeline of the binary classification consistency (BCC) regulation. The module compares global and local top-k confidence to identify the candidate and negative classes for each unlabeled sample. Moreover, it encourages consistent candidate-negative division between different perturbed views of the same sample, thereby introducing supervision signals for all unlabeled samples.
• Figure 4: Learning process visualization on CIFAR-10-40 (a-d) and STL-10-40 (e-h). The threshold for STL-10-40 is restricted within [0.9, 1.0] to mitigate the adverse effects of noisy pseudo-labels. In the analysis of SoftMatch, we employ $\mu_t-\sigma_t$ as its threshold. Samples with a confidence lower than $\mu_t-\sigma_t$ are assigned negligible weights, essentially treated as if they were discarded. Consequently, the class-average threshold of SoftMatch is $\mu_t-\sigma_t$, and its selected pseudo-label acc and utilization ratio can be computed like other threshold-based models. Here, $\mu_t / \sigma_t$ denotes the mean/std of the overall confidence on unlabeled data. Detailed analysis for SoftMatch is provided in Appendix A.3.
• Figure 5: Grid search for the weight of BCC regulation $\lambda_b$.