Live-Evo: Online Evolution of Agentic Memory from Continuous Feedback

TL;DR

Live-Evo tackles online, continuous evolution of agentic memory for LLM-based agents operating under non-stationary, streaming task distributions. It decouples what happened from how to use it via two memory banks—the Experience Bank and Meta-Guideline Bank—and a four-stage loop Retrieve-Compile-Act-Update that learns to transform experiences into task-adaptive guidance. The system employs ContrastiveEval to quantify the memory's contribution and reinforcement-decay weight updates, with selective write-back to control growth. Empirically, Live-Evo delivers substantial improvements on Prophet Arena (20.8% better Brier score; 12.9% higher market returns), generalizes to deep-research benchmarks, and ablation studies corroborate the necessity of its components for online performance.

Abstract

Large language model (LLM) agents are increasingly equipped with memory, which are stored experience and reusable guidance that can improve task-solving performance. Recent \emph{self-evolving} systems update memory based on interaction outcomes, but most existing evolution pipelines are developed for static train/test splits and only approximate online learning by folding static benchmarks, making them brittle under true distribution shift and continuous feedback. We introduce \textsc{Live-Evo}, an online self-evolving memory system that learns from a stream of incoming data over time. \textsc{Live-Evo} decouples \emph{what happened} from \emph{how to use it} via an Experience Bank and a Meta-Guideline Bank, compiling task-adaptive guidelines from retrieved experiences for each task. To manage memory online, \textsc{Live-Evo} maintains experience weights and updates them from feedback: experiences that consistently help are reinforced and retrieved more often, while misleading or stale experiences are down-weighted and gradually forgotten, analogous to reinforcement and decay in human memory. On the live \textit{Prophet Arena} benchmark over a 10-week horizon, \textsc{Live-Evo} improves Brier score by 20.8\% and increases market returns by 12.9\%, while also transferring to deep-research benchmarks with consistent gains over strong baselines. Our code is available at https://github.com/ag2ai/Live-Evo.

Figures (4)

• Figure 1: Traditional Self-Evolving Memory System build memory on training dataset and test with the evolved memory. While Live Self-Evolving Memory System build and learn to utilize Memory to tackle continuously new data.
• Figure 2: Structure of Live-Evo Agent. Given a question, the Live-Evo Agent will first search relevant experiences and generate a guideline based on the experiences, current task. Also, the generation will augmented by the meta-guideline bank, which teaches the agent how to combine experiences with current task. Inside the agent, the memory update mechanism continually updating experiences' weights and verifying new experiences and meta-guidelines.
• Figure 3: Performance Analysis Comparison. (a) shows the cumulative portfolio value, and (b) shows the Brier score comparison.
• Figure 4: Case Study. The figure contrasts a high-weight experience (green), which provides reusable methods, with a low-weight experience (red), which contains hallucinations, and shows how their weights evolve weekly.