Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Replay-Free Continual Low-Rank Adaptation with Dynamic Memory

Huancheng Chen, Jingtao Li, Weiming Zhuang, Chen Chen, Lingjuan Lyu

Abstract

We revisit continual learning~(CL), which enables pre-trained vision transformers (ViTs) to sequentially fine-tune on new downstream tasks over time. However, as the scale of these models increases, catastrophic forgetting remains a more serious challenge. Recent studies highlight a crossover between CL techniques and parameter-efficient fine-tuning (PEFT), which focuses on fine-tuning only a small set of trainable parameters to adapt to downstream tasks, such as low-rank adaptation (LoRA). While LoRA achieves faster convergence and requires fewer trainable parameters, it has seldom been explored in the context of continual learning. To address this gap, we propose a novel PEFT-CL method called Dual Low-Rank Adaptation (DualLoRA), which introduces both an orthogonal LoRA adapter and a residual LoRA adapter parallel to pre-trained weights in each layer. These components are orchestrated by a dynamic memory mechanism to strike a balance between stability and plasticity. Additionally, we propose a scheme to predict task identity with confidence and calibrate the model's outputs accordingly. On ViT-based models, we demonstrate that DualLoRA offers significant advantages in accuracy, inference speed, and computation efficiency in training over existing CL methods across multiple benchmarks.

Replay-Free Continual Low-Rank Adaptation with Dynamic Memory

Abstract

We revisit continual learning~(CL), which enables pre-trained vision transformers (ViTs) to sequentially fine-tune on new downstream tasks over time. However, as the scale of these models increases, catastrophic forgetting remains a more serious challenge. Recent studies highlight a crossover between CL techniques and parameter-efficient fine-tuning (PEFT), which focuses on fine-tuning only a small set of trainable parameters to adapt to downstream tasks, such as low-rank adaptation (LoRA). While LoRA achieves faster convergence and requires fewer trainable parameters, it has seldom been explored in the context of continual learning. To address this gap, we propose a novel PEFT-CL method called Dual Low-Rank Adaptation (DualLoRA), which introduces both an orthogonal LoRA adapter and a residual LoRA adapter parallel to pre-trained weights in each layer. These components are orchestrated by a dynamic memory mechanism to strike a balance between stability and plasticity. Additionally, we propose a scheme to predict task identity with confidence and calibrate the model's outputs accordingly. On ViT-based models, we demonstrate that DualLoRA offers significant advantages in accuracy, inference speed, and computation efficiency in training over existing CL methods across multiple benchmarks.

Paper Structure

This paper contains 20 sections, 19 equations, 4 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. Gradient Projection in CL
  4. Parameter-Efficient Fine-Tuning in CL
  5. Preliminary
  6. Continual Learning Problem Setting
  7. Multi-Head Attention Block
  8. Low-Rank Adaptation
  9. Methodology: Dual Low-Rank Adaptation
  10. Orthogonal Adapter
  11. Residual Adapter
  12. Dynamic Memory
  13. Task Identification with Confidence
  14. Experiments
  15. Experimental Settings
  16. ...and 5 more sections

Figures (4)

  • Figure 1: PEFT-based continual learning schemes dominate the ImageNet-R dataset in recent years.
  • Figure 2: Illustration of our proposed DualLoRA paradigm (left) and design insights of orthogonal adapter and residual adapter (right), where the solid arrow denotes the original update and the dashed arrow denotes the projected update.
  • Figure 3: Figures (a) and (b) demonstrate the average accuracy of different methods during training.
  • Figure 4: Figure (a) demonstrates the approximated average FLOPs during training and inference on each batch of data points. Figure (b) demonstrates the actual average running time for different schemes to perform inference on a task.