Continual Learning of Large Language Models: A Comprehensive Survey

TL;DR

This survey frames continual learning for large language models as a two-dimensional problem: vertical continuity (general-to-domain adaptation) and horizontal continuity (temporal-domain evolution). It surveys three learning stages—Continual Pre-Training, Domain-Adaptive Pre-training, and Continual Fine-Tuning—along with techniques (replay, regularization, architecture expansion), evaluation protocols, and domain-specific applications. Key contributions include a structured taxonomy, cross-domain observations, and a roadmap of benchmarks and future directions to guide efficient, reliable continual development of LLMs. The work aims to bridge CL theory with practical LLM deployment, highlighting memory, computation, and alignment considerations for real-world continual learning.

Abstract

The recent success of large language models (LLMs) trained on static, pre-collected, general datasets has sparked numerous research directions and applications. One such direction addresses the non-trivial challenge of integrating pre-trained LLMs into dynamic data distributions, task structures, and user preferences. Pre-trained LLMs, when tailored for specific needs, often experience significant performance degradation in previous knowledge domains -- a phenomenon known as "catastrophic forgetting". While extensively studied in the continual learning (CL) community, it presents new manifestations in the realm of LLMs. In this survey, we provide a comprehensive overview of the current research progress on LLMs within the context of CL. This survey is structured into four main sections: we first describe an overview of continually learning LLMs, consisting of two directions of continuity: vertical continuity (or vertical continual learning), i.e., continual adaptation from general to specific capabilities, and horizontal continuity (or horizontal continual learning), i.e., continual adaptation across time and domains (Section 3). We then summarize three stages of learning LLMs in the context of modern CL: Continual Pre-Training (CPT), Domain-Adaptive Pre-training (DAP), and Continual Fine-Tuning (CFT) (Section 4). Then we provide an overview of evaluation protocols for continual learning with LLMs, along with the current available data sources (Section 5). Finally, we discuss intriguing questions pertaining to continual learning for LLMs (Section 6). The full list of papers examined in this survey is available at https://github.com/Wang-ML-Lab/llm-continual-learning-survey.

Figures (2)

• Figure 1: A high-level overview of the modern pipeline for continually pre-training and fine-tuning LLMs, where two dimensions of continuity are described. Vertical Continuity (or Vertical Continual Learning): LLM training can be vertically divided into three stages: (i) Continual Pre-Training (CPT), (ii) Domain-Adaptive Pre-training (DAP), and (iii) Continual Fine-Tuning (CFT). The main focus is the retention of the LLM's general knowledge (prevention of vertical forgetting). Horizontal Continuity (or Horizontal Continual Learning): After the LLMs are deployed, the models are continually updated when a new set of data becomes available. The primary goal is to prevent horizontal forgetting in a long sequence of tasks.
• Figure 2: A diagram showing two different directions of continual learning of LLMs. (a) Vertical Continual Learning of LLMs: in this case, the upstream data distribution usually partially covers the subsequent tasks' data distribution. (b) Horizontal Continual Learning of LLMs: No constraints on the data distributions are present on horizontal continual learning. The continual LLMs need to handle the challenge of abrupt distributional shifts and a longer sequence of training.

Theorems & Definitions (9)

• Definition A.1: Instruction Tuning, IT
• Remark
• Definition A.2: Model Refinement, MR
• Definition A.3: Memory Constraint of Continual Learning
• Remark
• Definition A.4: Task-Incremental Learning, TIL
• Definition A.5: Domain-Incremental Learning, DIL
• Definition A.6: Class-Incremental Learning, CIL
• Remark