Lifelong Learning of Large Language Model based Agents: A Roadmap

TL;DR

This survey tackles the challenge of enabling large language model (LLM) agents to learn continually in dynamic environments. It introduces a three-module architecture—Perception, Memory, and Action—and formalizes lifelong learning in LLM agents via a goal-conditioned POMDP framework, outlining mechanisms to combat catastrophic forgetting through memory systems and retrieval strategies. The work reviews single- and multimodal perception, memory types (working, episodic, semantic, parametric), and action modalities (grounding, retrieval, reasoning), and surveys concrete techniques across environments (web, games) and applications (daily and domain-specific). It also provides evaluation metrics and benchmarks, discusses practical insights, and outlines future directions for robust, long-horizon planning in lifelong LLM agents. Collectively, the paper offers a comprehensive roadmap for integrating lifelong learning into LLM-based agents to achieve continuous, adaptive performance in real-world settings.

Abstract

Lifelong learning, also known as continual or incremental learning, is a crucial component for advancing Artificial General Intelligence (AGI) by enabling systems to continuously adapt in dynamic environments. While large language models (LLMs) have demonstrated impressive capabilities in natural language processing, existing LLM agents are typically designed for static systems and lack the ability to adapt over time in response to new challenges. This survey is the first to systematically summarize the potential techniques for incorporating lifelong learning into LLM-based agents. We categorize the core components of these agents into three modules: the perception module for multimodal input integration, the memory module for storing and retrieving evolving knowledge, and the action module for grounded interactions with the dynamic environment. We highlight how these pillars collectively enable continuous adaptation, mitigate catastrophic forgetting, and improve long-term performance. This survey provides a roadmap for researchers and practitioners working to develop lifelong learning capabilities in LLM agents, offering insights into emerging trends, evaluation metrics, and application scenarios. Relevant literature and resources are available at \href{this url}{https://github.com/qianlima-lab/awesome-lifelong-llm-agent}.

Figures (16)

• Figure 1: Number of publications on lifelong learning and LLM Agents (from Google Scholar). The publications have grown rapidly in recent three years.
• Figure 2: Comparison of lifelong learning between LLMs and LLM Agents. (a) Traditional lifelong learning paradigm of LLMs, where LLMs are viewed as static black-box systems without feedback from the environment; (b) the novel lifelong learning paradigm of LLM agents focused on in this survey, where agents interact with ever-changing environments. Please refer to Figure \ref{['fig:illustration_lifelong_learning_llm_agents']} for an illustration.
• Figure 3: A lifelong LLM agent can adapt to its environment and achieve behavioral evolution through interaction (adapted from AWM wang2024agent).
• Figure 4: Illustration of lifelong learning in large language model-based agents. In real-world applications, LLM agents are expected to adapt to various environments such as gaming, web browsing, shopping, household tasks, and operating systems without the need to design environment-specific agents for every new environment.
• Figure 5: Development of lifelong learning for AI systems, highlighting four key stages: (1) Establishment of foundational concepts starting in the 1980s, (2) Advancements in deep lifelong learning from 2010 to the present, (3) Integration of lifelong learning into large language models from 2020 onwards, and (4) the latest developments in lifelong learning for LLM agents. The practicality of lifelong learning has significantly improved alongside its development, enabling more versatile and adaptive AI systems in diverse real-world applications.

Theorems & Definitions (8)

• Definition 3.1: Environment of LLM Agents
• Definition 3.2: LLM-based Agent
• Definition 3.3: Task
• Definition 3.4: Trajectory
• Definition 3.5: Trial
• Definition 3.6: Trajectory-Level and Step-Level Rewards
• Definition 3.7: Lifelong Learning Tasks
• Definition 3.8: Objective of Lifelong Learning