SLCA++: Unleash the Power of Sequential Fine-tuning for Continual Learning with Pre-training

TL;DR

This work presents an in-depth analysis of the progressive overfitting problem from the lens of Seq FT, and introduces the advanced Slow Learner with Classifier Alignment (SLCA++) framework to unleash the power of Seq FT, serving as a strong baseline approach for CLPT.

Abstract

In recent years, continual learning with pre-training (CLPT) has received widespread interest, instead of its traditional focus of training from scratch. The use of strong pre-trained models (PTMs) can greatly facilitate knowledge transfer and alleviate catastrophic forgetting, but also suffers from progressive overfitting of pre-trained knowledge into specific downstream tasks. A majority of current efforts often keep the PTMs frozen and incorporate task-specific prompts to instruct representation learning, coupled with a prompt selection process for inference. However, due to the limited capacity of prompt parameters, this strategy demonstrates only sub-optimal performance in continual learning. In comparison, tuning all parameters of PTMs often provides the greatest potential for representation learning, making sequential fine-tuning (Seq FT) a fundamental baseline that has been overlooked in CLPT. To this end, we present an in-depth analysis of the progressive overfitting problem from the lens of Seq FT. Considering that the overly fast representation learning and the biased classification layer constitute this particular problem, we introduce the advanced Slow Learner with Classifier Alignment (SLCA++) framework to unleash the power of Seq FT, serving as a strong baseline approach for CLPT. Our approach involves a Slow Learner to selectively reduce the learning rate of backbone parameters, and a Classifier Alignment to align the disjoint classification layers in a post-hoc fashion. We further enhance the efficacy of SL with a symmetric cross-entropy loss, as well as employ a parameter-efficient strategy to implement Seq FT with SLCA++. Across a variety of continual learning scenarios on image classification benchmarks, our approach provides substantial improvements and outperforms state-of-the-art methods by a large margin. Code: https://github.com/GengDavid/SLCA.

Figures (8)

• Figure 1: Comparison of recent methods and our proposal. (a) Prompt-based methods (L2P wang2022l2p, DualPrompt wang2022dualprompt, CODA-Prompt smith2023coda, etc.) often construct and select appropriate prompt parameters for each task while keeping the backbone frozen. (b) LAE gao2023unified employs a momentum copy of additional parameters to stabilize their updates in CL. (c) Our proposal is implemented with the simplest baseline, i.e., sequential fine-tuning (Seq FT), unleashing its power with Slow Learner (SL) and Classifier Alignment (CA). (d) Evaluation of continual learning performance on Split CIFAR-100 with ImageNet-21K supervised pre-training.
• Figure 2: Slow Learner (SL) can greatly enhance the performance of sequential fine-tuning (Seq FT) in CLPT. Here we adopt ImageNet-21K supervised pre-training for all baselines with default performance referenced from previous efforts wang2022l2pwang2022dualprompt, including prompt-based methods (L2P wang2022l2p and DualPrompt wang2022dualprompt), regularization-based methods (EWC kirkpatrick2017overcoming and LwF li2017learning), and replay-based methods (GDumb prabhu2020gdumb, DER++ buzzega2020dark and BiC wu2019large).
• Figure 3: Comparison of pre-training paradigms on ImageNet-1K. DeiT touvron2021training is a strong supervised method for (pre-)training vision transformer, while MoCo v3 chen2021empirical, MAE he2022masked and BEiT bao2021beit are representative self-supervised methods. The pre-trained checkpoints are obtained from their official release.
• Figure 4: Similarity of the pre-trained representations (1) before and after joint training (left Y-axis, yellow dot), and (2) after joint training and after continual learning (right Y-axis, column). We adopt Centered Kernel Alignment (CKA) kornblith2019similarity as the similarity metric. Best viewed in color.
• Figure 5: Linear probing results of the Slow Learner. All experiments are based on ImageNet-21K supervised pre-training. We report the averaged accuracy of all classes in the corresponding benchmark dataset (e.g., a total of 100 classes in CIFAR-100 dataset). The dark red arrow represents the performance gap caused by a sub-optimal classification layer.