Learning from Few Examples: A Summary of Approaches to Few-Shot Learning

TL;DR

Few-Shot Learning tackles learning under data scarcity by leveraging meta-learning, transfer learning, and hybrids to enable rapid adaptation from a few examples. The paper surveys three meta-learning families—metric-based, optimization-based, and model-based—along with hybrid strategies and non-meta-learning approaches, detailing representative methods like Siamese/Matching/Prototypical networks, MAML and its variants, and memory-augmented architectures. It discusses standard episodic training, datasets, and empirical progress, highlighting substantial accuracy gains on miniImageNet and related benchmarks, while outlining practical challenges such as cross-domain generalization, generalized FSL, and domain shifts. Overall, the survey provides a comprehensive map of approaches enabling effective few-shot adaptation and points to open questions essential for robust, real-world deployment across domains.

Abstract

Few-Shot Learning refers to the problem of learning the underlying pattern in the data just from a few training samples. Requiring a large number of data samples, many deep learning solutions suffer from data hunger and extensively high computation time and resources. Furthermore, data is often not available due to not only the nature of the problem or privacy concerns but also the cost of data preparation. Data collection, preprocessing, and labeling are strenuous human tasks. Therefore, few-shot learning that could drastically reduce the turnaround time of building machine learning applications emerges as a low-cost solution. This survey paper comprises a representative list of recently proposed few-shot learning algorithms. Given the learning dynamics and characteristics, the approaches to few-shot learning problems are discussed in the perspectives of meta-learning, transfer learning, and hybrid approaches (i.e., different variations of the few-shot learning problem).

Figures (18)

• Figure 1: Approaches to FSL are categorized into Meta-Learning-based FSL and Non-Meta-Learning-based FSL. The three main meta-learning approaches are: metric-based, optimization-based and model-based meta-learning. Furthermore, variations of the FSL problem which use meta-learning are categorized as hybrid approaches.
• Figure 2: Meta-Learning Example Setup. Each task $\mathcal{T}_i$ is a binary classification task with a training set $\mathcal{D}^{train}_i$ and test set $\mathcal{D}^{test}_i$. During meta-training, the labels for samples in $\mathcal{D}^{test}_i$ is known and the goal of meta-learner is to find optimal $\theta$ as per equation \ref{['eq:meta-theta']}. During meta-testing, new task with unseen categories is presented and the labels are predicted as per equation \ref{['eq:meta-predict']} .
• Figure 3: Example metric-based meta-learning setup for a 4-way-1-shot classification task. The embedding function $g_{\theta_1}$ outputs the embedding vectors for support images (labeled) and the query image (unlabeled, denoted by '?'). Distance function $d_{\theta_2}$ measures the distance between support and query vectors to output a similarity score.
• Figure 4: Convolutional Siamese Network
• Figure 5: Matching Networks (Figure adapted from Matching2016).