Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Generalizable Two-Branch Framework for Image Class-Incremental Learning

Chao Wu, Xiaobin Chang, Ruixuan Wang

TL;DR

A novel two-branch continual learning framework is proposed to further enhance most existing CL methods and yields consistent improvements over state-of-the-art methods.

Abstract

Deep neural networks often severely forget previously learned knowledge when learning new knowledge. Various continual learning (CL) methods have been proposed to handle such a catastrophic forgetting issue from different perspectives and achieved substantial improvements. In this paper, a novel two-branch continual learning framework is proposed to further enhance most existing CL methods. Specifically, the main branch can be any existing CL model and the newly introduced side branch is a lightweight convolutional network. The output of each main branch block is modulated by the output of the corresponding side branch block. Such a simple two-branch model can then be easily implemented and learned with the vanilla optimization setting without whistles and bells. Extensive experiments with various settings on multiple image datasets show that the proposed framework yields consistent improvements over state-of-the-art methods.

Generalizable Two-Branch Framework for Image Class-Incremental Learning

TL;DR

A novel two-branch continual learning framework is proposed to further enhance most existing CL methods and yields consistent improvements over state-of-the-art methods.

Abstract

Deep neural networks often severely forget previously learned knowledge when learning new knowledge. Various continual learning (CL) methods have been proposed to handle such a catastrophic forgetting issue from different perspectives and achieved substantial improvements. In this paper, a novel two-branch continual learning framework is proposed to further enhance most existing CL methods. Specifically, the main branch can be any existing CL model and the newly introduced side branch is a lightweight convolutional network. The output of each main branch block is modulated by the output of the corresponding side branch block. Such a simple two-branch model can then be easily implemented and learned with the vanilla optimization setting without whistles and bells. Extensive experiments with various settings on multiple image datasets show that the proposed framework yields consistent improvements over state-of-the-art methods.
Paper Structure (7 sections, 1 equation, 3 figures, 7 tables)

This paper contains 7 sections, 1 equation, 3 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. The two-branch framework
  4. Experiments
  5. Main Results
  6. Detailed Analysis
  7. Conclusion

Figures (3)

  • Figure 1: The feature attention maps from different methods are illustrated. The 'Dog' class only appears in round 0 during continual learning. The models are then continually updated with other classes at different rounds. 'Finetune': model is simply finetuned across rounds. 'DER': a CL method. 'G2B(DER)': our G2B framework with DER.
  • Figure 2: The proposed two-branch framework for continual learning. The main branch (shown in yellow background) can be any continual learning framework with either CNN or Transformer backbone, and the side branch (shown in blue background) is a lightweight CNN. (a) The output of a main block is modulated by the corresponding side block. (b-c) Design of each side block and the adapter respectively for the CNN backbone and the ViT backbone of the main branch. Note that the BN and ReLU operators following the convolutional operator in each side block are omitted for simplicity.
  • Figure 3: Comparisons between the G2B framework and corresponding methods, DER (left) and DyTox (right), on CIFAR-100 under 10 rounds of continual learning.