Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Channel-ensemble Approach: Unbiased and Low-variance Pseudo-labels is Critical for Semi-supervised Classification

Jiaqi Wu, Junbiao Pang, Baochang Zhang, Qingming Huang

TL;DR

This work tackles the bias and high-variance issues of pseudo-labels in semi-supervised classification under scarce labels. It proposes Channel-Based Ensemble (CBE), a lightweight, plug-and-play multi-head ensemble with two branches, complemented by a Chebyshev constraint and two loss terms—Low Bias Loss and Low Variance Loss—to produce unbiased, low-variance pseudo-labels and stabilize self-training. Empirical results on CIFAR-10 and CIFAR-100 show that integrating CBE with FixMatch or FreeMatch yields substantial accuracy gains with minimal computational overhead, and ablations confirm the effectiveness of the LB and LV components. The method provides a general, efficient pathway to improve SSL PL quality across frameworks, with practical impact for low-label regimes and scalable deployment.

Abstract

Semi-supervised learning (SSL) is a practical challenge in computer vision. Pseudo-label (PL) methods, e.g., FixMatch and FreeMatch, obtain the State Of The Art (SOTA) performances in SSL. These approaches employ a threshold-to-pseudo-label (T2L) process to generate PLs by truncating the confidence scores of unlabeled data predicted by the self-training method. However, self-trained models typically yield biased and high-variance predictions, especially in the scenarios when a little labeled data are supplied. To address this issue, we propose a lightweight channel-based ensemble method to effectively consolidate multiple inferior PLs into the theoretically guaranteed unbiased and low-variance one. Importantly, our approach can be readily extended to any SSL framework, such as FixMatch or FreeMatch. Experimental results demonstrate that our method significantly outperforms state-of-the-art techniques on CIFAR10/100 in terms of effectiveness and efficiency.

A Channel-ensemble Approach: Unbiased and Low-variance Pseudo-labels is Critical for Semi-supervised Classification

TL;DR

This work tackles the bias and high-variance issues of pseudo-labels in semi-supervised classification under scarce labels. It proposes Channel-Based Ensemble (CBE), a lightweight, plug-and-play multi-head ensemble with two branches, complemented by a Chebyshev constraint and two loss terms—Low Bias Loss and Low Variance Loss—to produce unbiased, low-variance pseudo-labels and stabilize self-training. Empirical results on CIFAR-10 and CIFAR-100 show that integrating CBE with FixMatch or FreeMatch yields substantial accuracy gains with minimal computational overhead, and ablations confirm the effectiveness of the LB and LV components. The method provides a general, efficient pathway to improve SSL PL quality across frameworks, with practical impact for low-label regimes and scalable deployment.

Abstract

Semi-supervised learning (SSL) is a practical challenge in computer vision. Pseudo-label (PL) methods, e.g., FixMatch and FreeMatch, obtain the State Of The Art (SOTA) performances in SSL. These approaches employ a threshold-to-pseudo-label (T2L) process to generate PLs by truncating the confidence scores of unlabeled data predicted by the self-training method. However, self-trained models typically yield biased and high-variance predictions, especially in the scenarios when a little labeled data are supplied. To address this issue, we propose a lightweight channel-based ensemble method to effectively consolidate multiple inferior PLs into the theoretically guaranteed unbiased and low-variance one. Importantly, our approach can be readily extended to any SSL framework, such as FixMatch or FreeMatch. Experimental results demonstrate that our method significantly outperforms state-of-the-art techniques on CIFAR10/100 in terms of effectiveness and efficiency.
Paper Structure (18 sections, 11 equations, 6 figures, 4 tables)

This paper contains 18 sections, 11 equations, 6 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Strategies to reduce bias and variance
  4. Ensemble Learning
  5. Method
  6. Overview
  7. Chebyshev Constraint
  8. Low Bias Loss
  9. Low Variance Loss
  10. Total Loss
  11. Implementation Details
  12. Experiments
  13. Configurations
  14. Comparisons
  15. Ablation
  16. ...and 3 more sections

Figures (6)

  • Figure 1: Comparison the sampling rate (SR) and accuracy of the generated PLs between FixMatch/FreeMatch and ours on CIFAR-10. The proposed method significantly improves the quality of pseudo-labels, maintaining a low SR.
  • Figure 2: The comparisons among the different ensemble methods.
  • Figure 3: The neural network structure of our network.
  • Figure 4: The structure of Low Bias loss (LB loss).
  • Figure 5: Accuracy curves of PLs in ablation.
  • ...and 1 more figures