Pseudo-Labeling Based Practical Semi-Supervised Meta-Training for Few-Shot Learning

TL;DR

This work tackles the label bottleneck in meta-training for few-shot learning by introducing a practical semi-supervised meta-training setting with truly unlabeled data. It proposes Pseudo-Labeling Based Meta-Learning (PLML), a two-stage framework that pre-trains with SSL on labeled and unlabeled data and then uses pseudo-labels for unlabeled data to construct few-shot tasks, accompanied by a noise-suppressing finetuning mechanism based on a Learnable Smoothing Module and a Noise Dropout strategy. The approach is demonstrated across miniImageNet and tieredImageNet with multiple SSL baselines and backbones, yielding significant improvements over existing semi-supervised meta-training methods and narrowing the gap to fully labeled baselines, while also enhancing SSL performance through meta-learning. The results suggest PLML as a robust, model-agnostic interface between semi-supervised learning and meta-learning, with practical implications for deploying FSL in data-scarce or annotation-costly domains.

Abstract

Most existing few-shot learning (FSL) methods require a large amount of labeled data in meta-training, which is a major limit. To reduce the requirement of labels, a semi-supervised meta-training (SSMT) setting has been proposed for FSL, which includes only a few labeled samples and numbers of unlabeled samples in base classes. However, existing methods under this setting require class-aware sample selection from the unlabeled set, which violates the assumption of unlabeled set. In this paper, we propose a practical semi-supervised meta-training setting with truly unlabeled data to facilitate the applications of FSL in realistic scenarios. To better utilize both the labeled and truly unlabeled data, we propose a simple and effective meta-training framework, called pseudo-labeling based meta-learning (PLML). Firstly, we train a classifier via common semi-supervised learning (SSL) and use it to obtain the pseudo-labels of unlabeled data. Then we build few-shot tasks from labeled and pseudo-labeled data and design a novel finetuning method with feature smoothing and noise suppression to better learn the FSL model from noise labels. Surprisingly, through extensive experiments across two FSL datasets, we find that this simple meta-training framework effectively prevents the performance degradation of various FSL models under limited labeled data, and also significantly outperforms the state-of-the-art SSMT models. Besides, benefiting from meta-training, our method also improves two representative SSL algorithms as well.

Figures (4)

• Figure 1: Illustrations of performance degradation with the reduction of labels in few-shot learning (FSL). The horizontal and vertical axes represent the number of labels per class and accuracy on miniImageNet ravi2016optimization. We demonstrate the performance degradation of three representative Few-Shot Learning (FSL) models (inductive Proto (a), transductive T-EProdriguez2020embedding (b), and semi-supervised S-EProdriguez2020embedding (c)) when using different numbers of training labels in various tasks. "Full" denotes training with fully labeled data, while "PLML" indicates incorporation with our pseudo-labeling based meta-learning approach. In (d), we compare two models: SPNren2018meta, and M-PLli2022platinum, in both the original setting and our new setting (*-N).
• Figure 2: Existing semi-supervised meta-training task ren2018meta (black box) v.s. the proposed (yellow box). Green and blue blocks represent labeled and unlabeled data, respectively.
• Figure 3: The framework of our pseudo-labeling based meta-learning. Our framework contains two stages: pre-training with semi-supervised learning and fine-tuning with noise suppression.
• Figure 4: Accuracy (%) of SSL methods, including SemCo, FlexMatch, and our models: SemCo-in, SemCo-trans, FlexMatch-in, and FlexMatch-trans. in and trans represent inductive Proto and transductive EP, respectively.