Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Rehearsal-free and Task-free Online Continual Learning With Contrastive Prompt

Aopeng Wang, Ke Deng, Yongli Ren, Jun Luo

TL;DR

The paper tackles catastrophic forgetting in online continual learning when data arrive in a single pass without replay or task labels. It introduces Rehearsal-free and Task-free Online Continual Learning (F2OCL) by coupling class-specific prompts with a nearest-class-mean classifier on top of a frozen Vision Transformer encoder. A prompt pool of class-key-prompt triplets is learned via a prompt-contrastive objective, enabling augmented embeddings to cluster for the same class and separate across classes without storing past samples. Experimental results on CIFAR-100 and ImageNet-R show improved average final accuracy and reduced forgetting under rehearsal-free, task-free conditions, validating the approach's privacy-preserving and scalable potential.

Abstract

The main challenge of continual learning is \textit{catastrophic forgetting}. Because of processing data in one pass, online continual learning (OCL) is one of the most difficult continual learning scenarios. To address catastrophic forgetting in OCL, some existing studies use a rehearsal buffer to store samples and replay them in the later learning process, other studies do not store samples but assume a sequence of learning tasks so that the task identities can be explored. However, storing samples may raise data security or privacy concerns and it is not always possible to identify the boundaries between learning tasks in one pass of data processing. It motivates us to investigate rehearsal-free and task-free OCL (F2OCL). By integrating prompt learning with an NCM classifier, this study has effectively tackled catastrophic forgetting without storing samples and without usage of task boundaries or identities. The extensive experimental results on two benchmarks have demonstrated the effectiveness of the proposed method.

Rehearsal-free and Task-free Online Continual Learning With Contrastive Prompt

TL;DR

The paper tackles catastrophic forgetting in online continual learning when data arrive in a single pass without replay or task labels. It introduces Rehearsal-free and Task-free Online Continual Learning (F2OCL) by coupling class-specific prompts with a nearest-class-mean classifier on top of a frozen Vision Transformer encoder. A prompt pool of class-key-prompt triplets is learned via a prompt-contrastive objective, enabling augmented embeddings to cluster for the same class and separate across classes without storing past samples. Experimental results on CIFAR-100 and ImageNet-R show improved average final accuracy and reduced forgetting under rehearsal-free, task-free conditions, validating the approach's privacy-preserving and scalable potential.

Abstract

The main challenge of continual learning is \textit{catastrophic forgetting}. Because of processing data in one pass, online continual learning (OCL) is one of the most difficult continual learning scenarios. To address catastrophic forgetting in OCL, some existing studies use a rehearsal buffer to store samples and replay them in the later learning process, other studies do not store samples but assume a sequence of learning tasks so that the task identities can be explored. However, storing samples may raise data security or privacy concerns and it is not always possible to identify the boundaries between learning tasks in one pass of data processing. It motivates us to investigate rehearsal-free and task-free OCL (F2OCL). By integrating prompt learning with an NCM classifier, this study has effectively tackled catastrophic forgetting without storing samples and without usage of task boundaries or identities. The extensive experimental results on two benchmarks have demonstrated the effectiveness of the proposed method.

Paper Structure

This paper contains 15 sections, 8 equations, 5 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Works
  3. Problem Statement
  4. Background
  5. Methodology
  6. F2OCL Training
  7. Classification with Prompt-based NCM Classifier
  8. Experiments
  9. Overall Performance
  10. Forgetting Resistance
  11. Effect of Prompt
  12. Embedding Distribution
  13. Multiple Passes and Keys
  14. Conclusion
  15. Acknowledgements

Figures (5)

  • Figure 1: The proposed solution for F2OCL.
  • Figure 2: Contrastive prompt learning.
  • Figure 3: Forgetting resistance on CIFAR-100 (left) and on ImageNet-R (right) with weights DINO-1K.
  • Figure 4: The impact to our method when processing data by multiple passes and selecting $\mathcal{K}$ keys from the prompt pool.
  • Figure 5: The embeddings distribution of 10 samples of 10 classes (CIFAR-100).