Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Comprehensive Survey of Challenges and Opportunities of Few-Shot Learning Across Multiple Domains

Andrea Gajic, Sudip Vhaduri

TL;DR

This survey addresses the fundamental problem of learning under limited labeled data across audio, image, NLP, and multi-modal domains. It inventories and compares core few-shot methodologies—Model-Agnostic Meta-Learning, transfer learning, and dynamic/flexible approaches—alongside prompt-based, embedding-based, and data-augmentation strategies in NLP, and metric-learning in vision, with explicit attention to time and resource constraints. The paper highlights open challenges such as negative transfer, task-specific hyperparameter sensitivity, and computational demands, while identifying opportunities for cross-domain and multi-modal integration to improve generalization. By consolidating domain-specific strengths, limitations, and practical considerations, the work provides actionable guidance for researchers and decision-makers to deploy effective few-shot solutions in time-sensitive, data-scarce settings.

Abstract

In a world where new domains are constantly discovered and machine learning (ML) is applied to automate new tasks every day, challenges arise with the number of samples available to train ML models. While the traditional ML training relies heavily on data volume, finding a large dataset with a lot of usable samples is not always easy, and often the process takes time. For instance, when a new human transmissible disease such as COVID-19 breaks out and there is an immediate surge for rapid diagnosis, followed by rapid isolation of infected individuals from healthy ones to contain the spread, there is an immediate need to create tools/automation using machine learning models. At the early stage of an outbreak, it is not only difficult to obtain a lot of samples, but also difficult to understand the details about the disease, to process the data needed to train a traditional ML model. A solution for this can be a few-shot learning approach. This paper presents challenges and opportunities of few-shot approaches that vary across major domains, i.e., audio, image, text, and their combinations, with their strengths and weaknesses. This detailed understanding can help to adopt appropriate approaches applicable to different domains and applications.

A Comprehensive Survey of Challenges and Opportunities of Few-Shot Learning Across Multiple Domains

TL;DR

This survey addresses the fundamental problem of learning under limited labeled data across audio, image, NLP, and multi-modal domains. It inventories and compares core few-shot methodologies—Model-Agnostic Meta-Learning, transfer learning, and dynamic/flexible approaches—alongside prompt-based, embedding-based, and data-augmentation strategies in NLP, and metric-learning in vision, with explicit attention to time and resource constraints. The paper highlights open challenges such as negative transfer, task-specific hyperparameter sensitivity, and computational demands, while identifying opportunities for cross-domain and multi-modal integration to improve generalization. By consolidating domain-specific strengths, limitations, and practical considerations, the work provides actionable guidance for researchers and decision-makers to deploy effective few-shot solutions in time-sensitive, data-scarce settings.

Abstract

In a world where new domains are constantly discovered and machine learning (ML) is applied to automate new tasks every day, challenges arise with the number of samples available to train ML models. While the traditional ML training relies heavily on data volume, finding a large dataset with a lot of usable samples is not always easy, and often the process takes time. For instance, when a new human transmissible disease such as COVID-19 breaks out and there is an immediate surge for rapid diagnosis, followed by rapid isolation of infected individuals from healthy ones to contain the spread, there is an immediate need to create tools/automation using machine learning models. At the early stage of an outbreak, it is not only difficult to obtain a lot of samples, but also difficult to understand the details about the disease, to process the data needed to train a traditional ML model. A solution for this can be a few-shot learning approach. This paper presents challenges and opportunities of few-shot approaches that vary across major domains, i.e., audio, image, text, and their combinations, with their strengths and weaknesses. This detailed understanding can help to adopt appropriate approaches applicable to different domains and applications.

Paper Structure

This paper contains 32 sections, 15 equations, 10 figures.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Audio Domain
  4. Introduction to Few Shot Learning Approaches and Audio
  5. Few Shot Learning Approaches for Audio
  6. Model-Agnostic Meta-Learning
  7. Transfer Learning
  8. Dynamic Few Shot Learning
  9. Time and Resource Considerations
  10. Applications
  11. Image Domain
  12. Introduction to Few-Shot Learning Approaches and Image
  13. Few Shot Learning Approaches for Image
  14. Meta-Learning
  15. Fine-Tuning Transfer-Learning
  16. ...and 17 more sections

Figures (10)

  • Figure 1: Diagram of Model-Agnostic Meta-learning Application for Audio
  • Figure 2: Diagram of Transfer Learning Application for Audio
  • Figure 3: Diagram of Dynamic Few-Shot Learning Approach for Audio
  • Figure 4: Diagram of Meta-Learning Approach for Image Domain
  • Figure 5: Diagram of Fine-Tuning Transfer Learning Approach for Images
  • ...and 5 more figures