TEAL: New Selection Strategy for Small Buffers in Experience Replay Class Incremental Learning

TL;DR

The paper tackles catastrophic forgetting in class-incremental learning under tight replay-memory budgets by introducing TEAL, a typicality-based exemplar selection method that also ensures diversity via clustering in a learned embedding space. TEAL selects representative exemplars per class through an iterative, pace-driven process using K-Means to cover uncovered regions and a priority-removal scheme to preserve the most informative samples; it is designed as a modular add-on for existing ER-IL approaches. Across Split CIFAR-100, Split tinyImageNet, and Split CUB-200, TEAL yields state-of-the-art or near-state-of-the-art final accuracies, especially when the buffer is very small (1–3 exemplars per class), and remains beneficial across multiple architectures and task settings. The work provides practical impact for memory-constrained continual learning systems, and the authors provide code for reproducibility at their GitHub repository.

Abstract

Continual Learning is an unresolved challenge, whose relevance increases when considering modern applications. Unlike the human brain, trained deep neural networks suffer from a phenomenon called catastrophic forgetting, wherein they progressively lose previously acquired knowledge upon learning new tasks. To mitigate this problem, numerous methods have been developed, many relying on the replay of past exemplars during new task training. However, as the memory allocated for replay decreases, the effectiveness of these approaches diminishes. On the other hand, maintaining a large memory for the purpose of replay is inefficient and often impractical. Here we introduce TEAL, a novel approach to populate the memory with exemplars, that can be integrated with various experience-replay methods and significantly enhance their performance with small memory buffers. We show that TEAL enhances the average accuracy of existing class-incremental methods and outperforms other selection strategies, achieving state-of-the-art performance even with small memory buffers of 1-3 exemplars per class in the final task. This confirms our initial hypothesis that when memory is scarce, it is best to prioritize the most typical data. Code is available at this https URL: https://github.com/shahariel/TEAL.

Figures (12)

• Figure 1: Illustration of CIL with Experience Replay.
• Figure 2: Illustration of TEAL's iterative class selection process, which establishes a priority order for the selected set. Initially, an embedding space is generated separately for each class (shown on the left). Samples are then selected iteratively with $s_1=4$, $s_2=7$, and $s_3=9$ (see text for details). Each row on the right panel represents one iteration: the left image displays the $s_i$ clusters (obtained using K-means) with the previously selected points $S_{i-1}$ marked with 'o', while the right image shows the updated set $S_i$, with newly selected samples marked with 'X'. In the last iteration 3 clusters remain uncovered, but only 2 samples are selected, leaving the red cluster uncovered by $S_3$.
• Figure 3: Baseline CIL using Split CIFAR-100 with a small buffer size $|{\mathcal{M}}| = 300$. 7 baseline methods are shown, reporting the average accuracy $A_{t}$ as new tasks are learned.
• Figure 4: The performance of ER-IL methods with and without TEAL. First row displays the average accuracy after training incrementally on a different number of classes. Each color corresponds to a different ER-IL method, where the continuous line represents the vanilla method, while the dashed line represents the method with TEAL as its selection strategy. The error bars correspond to standard error based on 4-10 repetitions. Second row depicts the difference in accuracy between TEAL and another method (XDER, ER-ACE, and ER) across all tasks.
• Figure 5: Performance improvement of TEAL when integrated with XDER over various buffer sizes.

Theorems & Definitions (1)

• Definition 3.1: Typicality