Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Complete Survey on Contemporary Methods, Emerging Paradigms and Hybrid Approaches for Few-Shot Learning

Georgios Tsoumplekas, Vladislav Li, Panagiotis Sarigiannidis, Vasileios Argyriou

TL;DR

Few-Shot Learning (FSL) addresses rapid adaptation from scarce data by leveraging knowledge across related tasks. The paper presents a comprehensive taxonomy that extends traditional meta-learning with in-context learning, neural processes, and probabilistic meta-learning, and discusses hybrid approaches that extend FSL beyond supervised settings. It surveys foundational formulations, datasets, model families, data augmentation, and In-Context Learning, and articulates applications across vision, NLP, healthcare, and more, along with trends such as Green AI and foundation-model evaluation. The work highlights challenges in evaluation, distribution shifts, and modality generalization, and outlines future directions toward human-like learning and broadly generalizable FSL systems with practical impact.

Abstract

Despite the widespread success of deep learning, its intense requirements for vast amounts of data and extensive training make it impractical for various real-world applications where data is scarce. In recent years, Few-Shot Learning (FSL) has emerged as a learning paradigm that aims to address these limitations by leveraging prior knowledge to enable rapid adaptation to novel learning tasks. Due to its properties that highly complement deep learning's data-intensive needs, FSL has seen significant growth in the past few years. This survey provides a comprehensive overview of both well-established methods as well as recent advancements in the FSL field. The presented taxonomy extends previously proposed ones by incorporating emerging FSL paradigms, such as in-context learning, along with novel categories within the meta-learning paradigm for FSL, including neural processes and probabilistic meta-learning. Furthermore, a holistic overview of FSL is provided by discussing hybrid FSL approaches that extend FSL beyond the typically examined supervised learning setting. The survey also explores FSL's diverse applications across various domains. Finally, recent trends shaping the field, outstanding challenges, and promising future research directions are discussed.

A Complete Survey on Contemporary Methods, Emerging Paradigms and Hybrid Approaches for Few-Shot Learning

TL;DR

Few-Shot Learning (FSL) addresses rapid adaptation from scarce data by leveraging knowledge across related tasks. The paper presents a comprehensive taxonomy that extends traditional meta-learning with in-context learning, neural processes, and probabilistic meta-learning, and discusses hybrid approaches that extend FSL beyond supervised settings. It surveys foundational formulations, datasets, model families, data augmentation, and In-Context Learning, and articulates applications across vision, NLP, healthcare, and more, along with trends such as Green AI and foundation-model evaluation. The work highlights challenges in evaluation, distribution shifts, and modality generalization, and outlines future directions toward human-like learning and broadly generalizable FSL systems with practical impact.

Abstract

Despite the widespread success of deep learning, its intense requirements for vast amounts of data and extensive training make it impractical for various real-world applications where data is scarce. In recent years, Few-Shot Learning (FSL) has emerged as a learning paradigm that aims to address these limitations by leveraging prior knowledge to enable rapid adaptation to novel learning tasks. Due to its properties that highly complement deep learning's data-intensive needs, FSL has seen significant growth in the past few years. This survey provides a comprehensive overview of both well-established methods as well as recent advancements in the FSL field. The presented taxonomy extends previously proposed ones by incorporating emerging FSL paradigms, such as in-context learning, along with novel categories within the meta-learning paradigm for FSL, including neural processes and probabilistic meta-learning. Furthermore, a holistic overview of FSL is provided by discussing hybrid FSL approaches that extend FSL beyond the typically examined supervised learning setting. The survey also explores FSL's diverse applications across various domains. Finally, recent trends shaping the field, outstanding challenges, and promising future research directions are discussed.
Paper Structure (63 sections, 6 equations, 16 figures, 1 table)

This paper contains 63 sections, 6 equations, 16 figures, 1 table.

Table of Contents

  1. Introduction
  2. Background
  3. Problem Formulation
  4. N-way K-shot Formulation
  5. Datasets
  6. Relevant Learning Problems
  7. Few-Shot Learning Models
  8. Meta-Learning Approaches
  9. Metric-Based Methods
  10. Task-Invariant Representations
  11. Task-Specific Representations
  12. Model-Based Methods
  13. Architecture-Based Methods
  14. Memory-Based Methods
  15. Optimization-Based Methods
  16. ...and 48 more sections

Figures (16)

  • Figure 1: A taxonomy of FSL methods and their applications. This includes main methods focused on supervised FSL settings (Section \ref{['fsl_models']}), hybrid methods for FSL beyond the supervised setting (Section \ref{['beyond-supervised-FSL']}), and FSL fields of application (Section \ref{['fsl_applications']})
  • Figure 2: Matching networks architecture vinyals2016matching. Predictions are made using an attention mechanism to compare the representations of the support and query set samples
  • Figure 3: Prototypical networks architecture snell2017prototypical. A mean representation is calculated for each class, and the query sample is assigned to the class of the nearest centroid
  • Figure 4: Relation networks architecture sung2018learning. Support and query set representations are concatenated and classified based on their similarity following a metric learning approach
  • Figure 5: Memory-Augmented Neural Networks architecture santoro2016meta. An external memory module is utilized to store and retrieve refined sample representations within each task
  • ...and 11 more figures