Recent Deep Semi-supervised Learning Approaches and Related Works

TL;DR

This paper surveys deep semi-supervised learning (SSL) methods, addressing the problem of label scarcity by leveraging unlabeled data under foundational assumptions such as the manifold, continuity, and cluster hypotheses. It categorizes approaches into entropy regularization, self-training, consistency regularization (data-dependent and data-agnostic), and self-supervised strategies, and discusses holistic frameworks like MixMatch, RemixMatch, and FixMatch that unify these ideas. Key techniques include pseudo-labeling, Pi-models, Temporal Ensembling, Mean Teacher, NoisyStudent, MixUp variants, and Virtual Adversarial Training, with an emphasis on practical augmentation and training strategies. The paper also discusses limitations, notably distribution mismatch between labeled and unlabeled data, and suggests future work integrating generative models and more robust SSL paradigms for real-world applicability.

Abstract

This work proposes an overview of the recent semi-supervised learning approaches and related works. Despite the remarkable success of neural networks in various applications, there exist a few formidable constraints, including the need for a large amount of labeled data. Therefore, semi-supervised learning, which is a learning scheme in which scarce labels and a larger amount of unlabeled data are utilized to train models (e.g., deep neural networks), is getting more important. Based on the key assumptions of semi-supervised learning, which are the manifold assumption, cluster assumption, and continuity assumption, the work reviews the recent semi-supervised learning approaches. In particular, the methods in regard to using deep neural networks in a semi-supervised learning setting are primarily discussed. In addition, the existing works are first classified based on the underlying idea and explained, then the holistic approaches that unify the aforementioned ideas are detailed.