Pseudo-label Learning with Calibrated Confidence Using an Energy-based Model

TL;DR

This work tackles calibrated confidence in pseudo-labeling for semi-supervised image classification with very limited labeled data. It introduces EBPL, which jointly trains a neural-network classifier and an energy-based model sharing a feature extractor, enabling the classifier to consider the input data distribution and produce calibrated confidence through train-time learning. A multi-step pseudo-labeling scheme uses soft labels and a hybrid loss that couples classifier and EBM parameters, leading to higher accuracy and lower calibration error than a curriculum-labeling baseline, with soft labels providing additional calibration benefits. The approach is demonstrated on seven image datasets, showing practical improvements for robust semi-supervised learning and suggesting future use in outlier rejection and scaling to larger inputs.

Abstract

In pseudo-labeling (PL), which is a type of semi-supervised learning, pseudo-labels are assigned based on the confidence scores provided by the classifier; therefore, accurate confidence is important for successful PL. In this study, we propose a PL algorithm based on an energy-based model (EBM), which is referred to as the energy-based PL (EBPL). In EBPL, a neural network-based classifier and an EBM are jointly trained by sharing their feature extraction parts. This approach enables the model to learn both the class decision boundary and input data distribution, enhancing confidence calibration during network training. The experimental results demonstrate that EBPL outperforms the existing PL method in semi-supervised image classification tasks, with superior confidence calibration error and recognition accuracy.

Figures (8)

• Figure 1: Joint learning of a classifier and generative model and its benefit. (a) Training only the classifier results in wrongly high posterior probability near the class decision boundary. (b) Estimating the input distribution by the generative model allows us to consider the frequency of data occurrences. (c) Joint learning of the classifier and generative model helps to calibrate the confidence.
• Figure 2: Overview of the EBPL algorithm.
• Figure 3: Structure of the hybrid model employed in EBPL.
• Figure 4: Accuracy and ECE for each method when varying the number of labeled data for each class.
• Figure 5: Reliability diagram. Blue and red bars show the actual average accuracy in each bin and the gap to the ideal, respectively. The diagonal red dashed lines show the perfect calibration.