Self-Consolidation for Self-Evolving Agents

TL;DR

This work addresses the limitation of static, lifelong-learning LLM agents by introducing EvoSC, a dual-stage framework that combines non-parametric contrastive extraction of error-prone and successful experiences with parametric trajectory consolidation into learnable prompts. By distilling extensive textual histories into compact parameterized memory, EvoSC maintains context efficiency while preserving rich long-horizon knowledge, enabling robust test-time evolution. Empirical results on LifelongAgentBench across DB, OS, and KG domains show EvoSC achieving state-of-the-art performance and superior stability compared to existing trajectory replay methods, especially under longer interaction histories. The approach highlights a scalable, plug-and-play pathway to evolve LLM-based agents through lifelong interaction, with broad implications for resilient, autonomous AI systems.

Abstract

While large language model (LLM) agents have demonstrated impressive problem-solving capabilities, they typically operate as static systems, lacking the ability to evolve through lifelong interaction. Existing attempts to bridge this gap primarily rely on retrieving successful past trajectories as demonstrations. However, this paradigm faces two critical limitations. First, by focusing solely on success, agents overlook the rich pedagogical value embedded in failed attempts, preventing them from identifying and avoiding recurrent pitfalls. Second, continually accumulating textual experiences not only increases the time consumption during retrieval but also inevitably introduces noise and exhausts the largest context window of current LLMs. To address these challenges, we propose a novel self-evolving framework for LLM agents that introduces a complementary evolution mechanism: First, a contrastive reflection strategy is introduced to explicitly summarize error-prone patterns and capture reusable insights. Second, we propose a self-consolidation mechanism that distills non-parametric textual experience into compact learnable parameters. This enables the agent to internalize extensive historical experience directly into its latent space. Extensive experiments demonstrate the advantages of our method in long-term agent evolution.

Figures (12)

• Figure 1: Illustration of the proposed self-consolidation framework for LLM agents lifelong evolution.
• Figure 2: Performance comparison of different methods across the (a) Database (DB), (b) Operating System (OS), and (c) Knowledge Graph (KG) agent lifelong learning benchmarks using Llama 3.1-8B-Instruct dubey2024llama and Qwen 2.5-7B-Instruct qwen2. EvoSC surpasses strong baselines such as AWM wang2024agent, TER zheng2025lifelongagentbench, SCM wang2025scm, and A-MEM xu2025mem.
• Figure 3: Pipeline of EvoSC. The framework orchestrates a dual-memory system: (Left) Non-parametric contrastive extraction draws explicit error-prone and successful insights from past trajectories to guide immediate reasoning; (Right) Parametric trajectory consolidation internalizes extensive historical knowledge into compact learnable prompts via trajectory distillation, which effectively resolves the context-length explosion. This hybrid approach ensures scalable lifelong learning by leveraging both textual experience and parametric long-term knowledge.
• Figure 4: Contrastive prompt and knowledge example for KG dataset.
• Figure 5: Correct count comparison for dataset database (window size=100, 1 trajectory)