Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ConPET: Continual Parameter-Efficient Tuning for Large Language Models

Chenyang Song, Xu Han, Zheni Zeng, Kuai Li, Chen Chen, Zhiyuan Liu, Maosong Sun, Tao Yang

TL;DR

Inspired by the success of parameter-efficient tuning (PET), ConPET is proposed, a generalizable paradigm for continual task adaptation of LLMs with task-number-independent training complexity and helps multiple former methods reduce the scale of tunable parameters by over 3,000 times.

Abstract

Continual learning necessitates the continual adaptation of models to newly emerging tasks while minimizing the catastrophic forgetting of old ones. This is extremely challenging for large language models (LLMs) with vanilla full-parameter tuning due to high computation costs, memory consumption, and forgetting issue. Inspired by the success of parameter-efficient tuning (PET), we propose Continual Parameter-Efficient Tuning (ConPET), a generalizable paradigm for continual task adaptation of LLMs with task-number-independent training complexity. ConPET includes two versions with different application scenarios. First, Static ConPET can adapt former continual learning methods originally designed for relatively smaller models to LLMs through PET and a dynamic replay strategy, which largely reduces the tuning costs and alleviates the over-fitting and forgetting issue. Furthermore, to maintain scalability, Dynamic ConPET adopts separate PET modules for different tasks and a PET module selector for dynamic optimal selection. In our extensive experiments, the adaptation of Static ConPET helps multiple former methods reduce the scale of tunable parameters by over 3,000 times and surpass the PET-only baseline by at least 5 points on five smaller benchmarks, while Dynamic ConPET gains its advantage on the largest dataset. The codes and datasets are available at https://github.com/Raincleared-Song/ConPET.

ConPET: Continual Parameter-Efficient Tuning for Large Language Models

TL;DR

Inspired by the success of parameter-efficient tuning (PET), ConPET is proposed, a generalizable paradigm for continual task adaptation of LLMs with task-number-independent training complexity and helps multiple former methods reduce the scale of tunable parameters by over 3,000 times.

Abstract

Continual learning necessitates the continual adaptation of models to newly emerging tasks while minimizing the catastrophic forgetting of old ones. This is extremely challenging for large language models (LLMs) with vanilla full-parameter tuning due to high computation costs, memory consumption, and forgetting issue. Inspired by the success of parameter-efficient tuning (PET), we propose Continual Parameter-Efficient Tuning (ConPET), a generalizable paradigm for continual task adaptation of LLMs with task-number-independent training complexity. ConPET includes two versions with different application scenarios. First, Static ConPET can adapt former continual learning methods originally designed for relatively smaller models to LLMs through PET and a dynamic replay strategy, which largely reduces the tuning costs and alleviates the over-fitting and forgetting issue. Furthermore, to maintain scalability, Dynamic ConPET adopts separate PET modules for different tasks and a PET module selector for dynamic optimal selection. In our extensive experiments, the adaptation of Static ConPET helps multiple former methods reduce the scale of tunable parameters by over 3,000 times and surpass the PET-only baseline by at least 5 points on five smaller benchmarks, while Dynamic ConPET gains its advantage on the largest dataset. The codes and datasets are available at https://github.com/Raincleared-Song/ConPET.
Paper Structure (25 sections, 2 equations, 3 figures, 8 tables, 1 algorithm)

This paper contains 25 sections, 2 equations, 3 figures, 8 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Parameter-Efficient Tuning
  4. Continual Learning for LLMs
  5. Mixture-of-Expert for LLMs
  6. Proposed Methods
  7. Task Definition
  8. Static ConPET
  9. PET Adaptation and Example Encoding
  10. Dynamic Sampling Strategy
  11. Dynamic ConPET
  12. PET Module Pre-Selection
  13. Prediction with Active PET Modules
  14. Detailed Algorithm and Complexity Analysis
  15. Cache System
  16. ...and 10 more sections

Figures (3)

  • Figure 1: The comparison between traditional EMR parisi2019continual and adapted EMR with Static ConPET. The latter adopts the dynamic replay strategy for training data generation and PET for LLM tuning.
  • Figure 2: The architecture of ConPET when the number of active PET modules is 2. The working process can be split into two procedures: PET module pre-selection and prediction with active PET modules. All logits generated by a specific PET module will be saved instantly by the cache system after this module completes tuning.
  • Figure 3: The average accuracies (%) of different settings at each step throughout the learning process.