Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Active Few-Shot Learning for Text Classification

Saeed Ahmadnia, Arash Yousefi Jordehi, Mahsa Hosseini Khasheh Heyran, Seyed Abolghasem Mirroshandel, Owen Rambow, Cornelia Caragea

TL;DR

Active Few-Shot Learning for Text Classification introduces an iterative active-learning pipeline that selects $M$ unlabeled samples per iteration from a pool using hybrid uncertainty and representativeness criteria, augmenting the $K$-sized support set during fine-tuning of large language models. It leverages two embedding streams, En (encoder-based) and Sc (score-based), and five sampling strategies (Random, Rep, Un, UnRep, ClUn) to balance uncertainty, diversity, and representativeness. Evaluated on five tasks with BART and FLAN-T5, the method consistently outperforms random and non-iterative baselines, with FLAN-T5-Rep(En)-ClUn(En) often providing the strongest gains; in-context learning generally lags behind iterative fine-tuning but offers complementary insights. The work demonstrates meaningful performance gains with a transparent, open-source implementation and discusses practical trade-offs between accuracy and efficiency, highlighting opportunities for future enhancements such as additional embeddings, semi-supervised extensions, and multilingual extension.

Abstract

The rise of Large Language Models (LLMs) has boosted the use of Few-Shot Learning (FSL) methods in natural language processing, achieving acceptable performance even when working with limited training data. The goal of FSL is to effectively utilize a small number of annotated samples in the learning process. However, the performance of FSL suffers when unsuitable support samples are chosen. This problem arises due to the heavy reliance on a limited number of support samples, which hampers consistent performance improvement even when more support samples are added. To address this challenge, we propose an active learning-based instance selection mechanism that identifies effective support instances from the unlabeled pool and can work with different LLMs. Our experiments on five tasks show that our method frequently improves the performance of FSL. We make our implementation available on GitHub.

Active Few-Shot Learning for Text Classification

TL;DR

Active Few-Shot Learning for Text Classification introduces an iterative active-learning pipeline that selects unlabeled samples per iteration from a pool using hybrid uncertainty and representativeness criteria, augmenting the -sized support set during fine-tuning of large language models. It leverages two embedding streams, En (encoder-based) and Sc (score-based), and five sampling strategies (Random, Rep, Un, UnRep, ClUn) to balance uncertainty, diversity, and representativeness. Evaluated on five tasks with BART and FLAN-T5, the method consistently outperforms random and non-iterative baselines, with FLAN-T5-Rep(En)-ClUn(En) often providing the strongest gains; in-context learning generally lags behind iterative fine-tuning but offers complementary insights. The work demonstrates meaningful performance gains with a transparent, open-source implementation and discusses practical trade-offs between accuracy and efficiency, highlighting opportunities for future enhancements such as additional embeddings, semi-supervised extensions, and multilingual extension.

Abstract

The rise of Large Language Models (LLMs) has boosted the use of Few-Shot Learning (FSL) methods in natural language processing, achieving acceptable performance even when working with limited training data. The goal of FSL is to effectively utilize a small number of annotated samples in the learning process. However, the performance of FSL suffers when unsuitable support samples are chosen. This problem arises due to the heavy reliance on a limited number of support samples, which hampers consistent performance improvement even when more support samples are added. To address this challenge, we propose an active learning-based instance selection mechanism that identifies effective support instances from the unlabeled pool and can work with different LLMs. Our experiments on five tasks show that our method frequently improves the performance of FSL. We make our implementation available on GitHub.

Paper Structure

This paper contains 30 sections, 2 equations, 8 figures, 13 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Active Few-Shot Learning
  4. Embedding Methods
  5. Encoder (En):
  6. Scores (Sc):
  7. Sampling Methods
  8. Random:
  9. Representative (Rep):
  10. Uncertainty (Un):
  11. Uncertainty Representative (UnRep):
  12. Cluster Uncertainty (ClUn):
  13. Experimental Setup
  14. Datasets
  15. Fine-Tuning Experiments
  16. ...and 15 more sections

Figures (8)

  • Figure 1: Pipeline of the $i^{th}$ iteration in our approach. Yellow boxes represent different phases of the method. Blue boxes are constant inputs during all iterations. Red boxes are carried over and modified during all consecutive iterations. Green boxes are products of the current iteration that will not be used later in the approach.
  • Figure 2: The range (min and max) and average of micro-F1 (%) scores of all tasks with steps of 10 over five runs.
  • Figure 3: The range (min and max) and average of macro-F1 (%) scores of AG News and Amazon Reviews tasks with steps of 16 over five runs.
  • Figure 4: Prompt templates for MPQA Type task. The upper box shows a sample for zero-shot learning, and the lower one shows a sample for $K$-shot learning.
  • Figure 5: Prompt templates for MPQA Polarity task. The upper box shows a sample for zero-shot learning, and the lower one shows a sample for $K$-shot learning.
  • ...and 3 more figures