Active Self-Semi-Supervised Learning for Few Labeled Samples

TL;DR

This paper proposes a simple yet effective framework, active self-semi-supervised learning (AS3L), which bootstraps semi-supervised models with prior pseudo-labels (PPL), and develops active learning and label propagation strategies to obtain accurate PPL.

Abstract

Training deep models with limited annotations poses a significant challenge when applied to diverse practical domains. Employing semi-supervised learning alongside the self-supervised model offers the potential to enhance label efficiency. However, this approach faces a bottleneck in reducing the need for labels. We observed that the semi-supervised model disrupts valuable information from self-supervised learning when only limited labels are available. To address this issue, this paper proposes a simple yet effective framework, active self-semi-supervised learning (AS3L). AS3L bootstraps semi-supervised models with prior pseudo-labels (PPL). These PPLs are obtained by label propagation over self-supervised features. Based on the observations the accuracy of PPL is not only affected by the quality of features but also by the selection of the labeled samples. We develop active learning and label propagation strategies to obtain accurate PPL. Consequently, our framework can significantly improve the performance of models in the case of limited annotations while demonstrating fast convergence. On the image classification tasks across four datasets, our method outperforms the baseline by an average of 5.4\%. Additionally, it achieves the same accuracy as the baseline method in about 1/3 of the training time.

Figures (11)

• Figure 1: Pipeline of AS3L (Ours) and Existing Self-Semi-Supervised Learning Approache (SelfMatch) kim2021selfmatch. (a) SelfMatch involves self-supervised pre-training followed by semi-supervised fine-tuning, relying on weight initialization to benefit semi-supervised learning from self-supervised pre-training. (b) Beyond weight initialization, AS3L (ours) improves semi-supervised learning by selecting labeled samples and generating prior pseudo-labels based on self-supervised features, providing a better starting point for subsequent semi-supervised training.
• Figure 2: Consistency of sample labels with neighboring sample labels in self-supervised and semi-supervised feature spaces. The model was initialized with the self-supervised pre-training weights and then further trained using FlexMatch.
• Figure 3: Test accuracy of the semi-supervised models initialized with self-supervised pre-training and random initialization. Specifically, semi-supervised training utilizing 10 labeled samples on CIFAR-10. Semi-supervised training method employed: FlexMatch.
• Figure 4: The framework of our Active Self-Semi-Supervised learning(AS3L). AS3L consists of four components: (1) Obtaining self-supervised feature $f_{self}$ follows chen2021exploring; (2) Selecting labeled samples based on $f_{self}$ (Sec. \ref{['sec:ss_active']}); (3) Label Propagation based on clusters to get PPL $y_{prior}$ (Sec. \ref{['sec:prior_pslabel']}); (4) Semi-supervised training guided by $y_{prior}$ (Sec. \ref{['sec:semisup_training']}).
• Figure 5: T-SNE visualization of semi-supervised and self-supervised features, where semi-supervised features are trained with 40 labels on CIFAR-10 followed our method. Self-supervised features seem to be more loosely clustered, and the boundaries between clusters are not as well defined.