EVCL: Elastic Variational Continual Learning with Weight Consolidation

TL;DR

Elastic Variational Continual Learning with Weight Consolidation is introduced, a novel hybrid model that integrates the variational posterior approximation mechanism of Variational Continual Learning with the regularization-based parameter-protection strategy of Elastic Weight Consolidation.

Abstract

Continual learning aims to allow models to learn new tasks without forgetting what has been learned before. This work introduces Elastic Variational Continual Learning with Weight Consolidation (EVCL), a novel hybrid model that integrates the variational posterior approximation mechanism of Variational Continual Learning (VCL) with the regularization-based parameter-protection strategy of Elastic Weight Consolidation (EWC). By combining the strengths of both methods, EVCL effectively mitigates catastrophic forgetting and enables better capture of dependencies between model parameters and task-specific data. Evaluated on five discriminative tasks, EVCL consistently outperforms existing baselines in both domain-incremental and task-incremental learning scenarios for deep discriminative models.

Figures (5)

• Figure 1: Test set average accuracy over PermutedMNIST for EVCL and baseline models.
• Figure 2: Test set average accuracy over SplitMNIST for EVCL and baseline models.
• Figure 3: Test set average accuracy over SplitNotMNIST for EVCL and baseline models.
• Figure 4: Test set average accuracy over SplitFashionMNIST for EVCL and baseline models.
• Figure 5: Test set average accuracy over SplitCIFAR-10 for EVCL and baseline models.