FETCH: A Memory-Efficient Replay Approach for Continual Learning in Image Classification

TL;DR

The paper addresses catastrophic forgetting in class-incremental continual learning under tight memory constraints. It presents FETCH, a memory-efficient two-stage approach that freezes a pre-trained encoder and compresses replay exemplars so only the classification head requires retraining. Empirical results on CIFAR-10 and CIFAR-100 show that simple compression methods can outperform deep autoencoders and that pre-training plus fixed-encoder design yields noticeable gains, establishing FETCH as a competitive baseline for memory-based continual learning. The approach offers practical benefits for memory-constrained settings and suggests broader applicability as a baseline for compressed replay learning.

Abstract

Class-incremental continual learning is an important area of research, as static deep learning methods fail to adapt to changing tasks and data distributions. In previous works, promising results were achieved using replay and compressed replay techniques. In the field of regular replay, GDumb achieved outstanding results but requires a large amount of memory. This problem can be addressed by compressed replay techniques. The goal of this work is to evaluate compressed replay in the pipeline of GDumb. We propose FETCH, a two-stage compression approach. First, the samples from the continual datastream are encoded by the early layers of a pre-trained neural network. Second, the samples are compressed before being stored in the episodic memory. Following GDumb, the remaining classification head is trained from scratch using only the decompressed samples from the reply memory. We evaluate FETCH in different scenarios and show that this approach can increase accuracy on CIFAR10 and CIFAR100. In our experiments, simple compression methods (e.g., quantization of tensors) outperform deep autoencoders. In the future, FETCH could serve as a baseline for benchmarking compressed replay learning in constrained memory scenarios.

Figures (6)

• Figure 1: Overview of the proposed method: GDumb Prabhu20 (red arrows) saves samples from the continual datastream in the episodic memory without preprocessing. A neural network is trained from scratch using only the samples from the memory. FETCH (blue arrows) fixes the early layers of the neural network after pre-training. The samples are saved efficiently using the compressor and decompressor. Only the classification head needs to be retrained.
• Figure 2: Tradeoff between storage and performance. Lower is better.
• Figure 3: Effect of pre-training early layers. The annotations correspond to the number of blocks in the encoder. 0 is the standard GDumb configuration, while 3 means that only the parameters of the last block were updated.
• Figure 4: Effect of compressing exemplars. Note that a higher compression parameter means less compression except for thinning, where a higher parameter means more compression. The vertical lines show the baselines without compression.
• Figure 5: Performance on a fixed memory budget