Learning to Learn for Few-shot Continual Active Learning

TL;DR

This work addresses few-shot continual active learning (CAL) for NLP by marrying meta-learning with active data acquisition and memory replay. It introduces Meta-CAL, a MAML-based framework that learns a favorable initialization and uses memory-based meta-objectives coupled with consistency regularization to mitigate catastrophic forgetting while adapting quickly to new tasks under tight annotation budgets. Extensive experiments on five text classification datasets show Meta-CAL achieves competitive accuracy with far fewer labeled samples (e.g., 2000 unlabeled pool with 500–2000 labels per task) and that random sampling often provides robust generalization, highlighting the role of randomness in balancing stability and plasticity. The approach demonstrates practical potential for resource-constrained continual learning in NLP and offers insights into augmentation and memory strategies that support generalization across tasks and domains.

Abstract

Continual learning strives to ensure stability in solving previously seen tasks while demonstrating plasticity in a novel domain. Recent advances in continual learning are mostly confined to a supervised learning setting, especially in NLP domain. In this work, we consider a few-shot continual active learning setting where labeled data are inadequate, and unlabeled data are abundant but with a limited annotation budget. We exploit meta-learning and propose a method, called Meta-Continual Active Learning. This method sequentially queries the most informative examples from a pool of unlabeled data for annotation to enhance task-specific performance and tackle continual learning problems through meta-objective. Specifically, we employ meta-learning and experience replay to address inter-task confusion and catastrophic forgetting. We further incorporate textual augmentations to avoid memory over-fitting caused by experience replay and sample queries, thereby ensuring generalization. We conduct extensive experiments on benchmark text classification datasets from diverse domains to validate the feasibility and effectiveness of meta-continual active learning. We also analyze the impact of different active learning strategies on various meta continual learning models. The experimental results demonstrate that introducing randomness into sample selection is the best default strategy for maintaining generalization in meta-continual learning framework.

Figures (5)

• Figure 1: Per task accuracy at different learning stages. The dark color indicates high accuracy. From left to right, the color for each task progressively fades away, indicating forgetting happens while learning more tasks. Note that Yelp and Amazon are from the same domain (sentiment analysis).UNC shows a lighter color compared to other AL methods, indicating a higher degree of forgetting. The red-outlined box shows the accuracy on AGNews (Task 2) after learning DBpedia (Task 3).
• Figure 2: BWT and FWT for different AL strategies at each learning stage.
• Figure 3: T-SNE visualization of memory samples at different learning stages using training set order Yelp $\rightarrow$ AGNews $\rightarrow$ DBpedia $\rightarrow$ Amazon $\rightarrow$ Yahoo. The black-circled data points belong to the last task. Data points with darker colors represent samples from earlier tasks, except (d) after learning Task 4. Task 4 has the same domain as Task 1. Hence, data points with darker colors are belong to the latest task in (d).
• Figure 4: T-SNE visualization of memory samples with different AL strategies. A equal dispersion of the data indicates a good memory representation. We provide accuracy for a better comparison. Data points with darker colors (purple, violet and pink) represent samples from earlier tasks. The black-circled data points belong to the last task. The clustering of memory samples from the last task suggests model focuses more on inter-task generalization rather than on intra-task generalization.
• Figure 5: Performance on different annotation budgets.