Improving Semi-Supervised Contrastive Learning via Entropy-Weighted Confidence Integration of Anchor-Positive Pairs

TL;DR

This paper tackles semi-supervised contrastive learning under limited labeled data by introducing entropy-based confidence weighting. It proposes three components: a novel loss $L_{ ext{SSC-E}}$ with geometric-mean weighting for anchor–positive pairs, entropy-based sample selection, and entropy-based adaptive weighting to modulate sample contributions based on prediction uncertainty. The approach leverages $H(p(oldsymbol{z}_w^i))$ with thresholds $H_{ ext{base}}$ and $e_{ ext{min}}$ to decide how unlabeled samples contribute, while normalizing and weighting with $w_i$ and $ar{oldsymbol{\lambda}}$. Experiments on CIFAR-10/100 show improved accuracy and stability under low-label conditions, though gains are dataset-dependent and generalizability requires broader validation.

Abstract

Conventional semi-supervised contrastive learning methods assign pseudo-labels only to samples whose highest predicted class probability exceeds a predefined threshold, and then perform supervised contrastive learning using those selected samples. In this study, we propose a novel loss function that estimates the confidence of each sample based on the entropy of its predicted probability distribution and applies confidence-based adaptive weighting. This approach enables pseudo-label assignment even to samples that were previously excluded from training and facilitates contrastive learning that accounts for the confidence of both anchor and positive samples in a more principled manner. Experimental results demonstrate that the proposed method improves classification accuracy and achieves more stable learning performance even under low-label conditions.