Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Latest Advancements Towards Catastrophic Forgetting under Data Scarcity: A Comprehensive Survey on Few-Shot Class Incremental Learning

M. Anwar Ma'sum, Mahardhika Pratama, Igor Skrjanc

TL;DR

This survey addresses the problem of catastrophic forgetting in data-scarce, sequential learning scenarios by surveying few-shot class incremental learning (FSCIL) methods. It systematically catalogs five families of FSCIL approaches—backbone tuning, meta-learning, prototype-tuning, dynamic architectures, and parameter-efficient fine-tuning (PEFT)—and highlights the complementary roles of pre-trained models and language-guided mechanisms. The paper formalizes objective functions, analyzes the importance of prototype rectification, and assesses trends across backbones, classifiers, augmentation, and metrics, while outlining open challenges and actionable future directions (e.g., federated, online, open-world FSCIL). By synthesizing performance patterns and practical considerations, the work informs both method design and evaluation in real-world, data-scarce continual learning settings, with emphasis on stability-plasticity and the effective use of foundation models. This positions FSCIL research to leverage PEFT and language-guided learning to better handle data scarcity and distributional shifts in open environments. $| T|$

Abstract

Data scarcity significantly complicates the continual learning problem, i.e., how a deep neural network learns in dynamic environments with very few samples. However, the latest progress of few-shot class incremental learning (FSCIL) methods and related studies show insightful knowledge on how to tackle the problem. This paper presents a comprehensive survey on FSCIL that highlights several important aspects i.e. comprehensive and formal objectives of FSCIL approaches, the importance of prototype rectifications, the new learning paradigms based on pre-trained model and language-guided mechanism, the deeper analysis of FSCIL performance metrics and evaluation, and the practical contexts of FSCIL in various areas. Our extensive discussion presents the open challenges, potential solutions, and future directions of FSCIL.

Latest Advancements Towards Catastrophic Forgetting under Data Scarcity: A Comprehensive Survey on Few-Shot Class Incremental Learning

TL;DR

This survey addresses the problem of catastrophic forgetting in data-scarce, sequential learning scenarios by surveying few-shot class incremental learning (FSCIL) methods. It systematically catalogs five families of FSCIL approaches—backbone tuning, meta-learning, prototype-tuning, dynamic architectures, and parameter-efficient fine-tuning (PEFT)—and highlights the complementary roles of pre-trained models and language-guided mechanisms. The paper formalizes objective functions, analyzes the importance of prototype rectification, and assesses trends across backbones, classifiers, augmentation, and metrics, while outlining open challenges and actionable future directions (e.g., federated, online, open-world FSCIL). By synthesizing performance patterns and practical considerations, the work informs both method design and evaluation in real-world, data-scarce continual learning settings, with emphasis on stability-plasticity and the effective use of foundation models. This positions FSCIL research to leverage PEFT and language-guided learning to better handle data scarcity and distributional shifts in open environments.

Abstract

Data scarcity significantly complicates the continual learning problem, i.e., how a deep neural network learns in dynamic environments with very few samples. However, the latest progress of few-shot class incremental learning (FSCIL) methods and related studies show insightful knowledge on how to tackle the problem. This paper presents a comprehensive survey on FSCIL that highlights several important aspects i.e. comprehensive and formal objectives of FSCIL approaches, the importance of prototype rectifications, the new learning paradigms based on pre-trained model and language-guided mechanism, the deeper analysis of FSCIL performance metrics and evaluation, and the practical contexts of FSCIL in various areas. Our extensive discussion presents the open challenges, potential solutions, and future directions of FSCIL.

Paper Structure

This paper contains 22 sections, 7 equations, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Problem Formulation
  4. Types of FSCIL Setings
  5. FSCIL Approaches
  6. Backbone Tuning Approach
  7. Meta-Learning Approach
  8. Prototype-Tuning Approach
  9. Dynamic Architecture Approach
  10. Parameter Efficient Fine Tuning (PEFT) Approach
  11. Trends and Finding
  12. Backbone, Parameters, and Pre-trained Model (PTM)
  13. Prototype-based over Network-based Classifiers
  14. Image vs Feature Augmentation
  15. The Importance of Prototype Rectification
  16. ...and 7 more sections

Figures (3)

  • Figure 1: Types of FSCIL settings and their properties (a) Fully-Supervised, (b) Semi-Supervised (c) Unsupervised (d) Complete (e) Cross-Domain FSCIL
  • Figure 2: Visualization of FSCIL Approaches i.e. (a) backbone tuning approach, (b) meta learning approach, (c) prototype tuning approach, (d) dynamic structure approach, and (e) parameter efficient fine tuning (PEFT) approach.
  • Figure 3: Biased Prototype, Optimal Prototypes, and Types of Prototype Rectification Mechanism (a) Biased and Optimal Prototype (b) Rectification by Loss function only (c) by pseudo-prototypes (d) by training-free calibration (e) by trainable networks