DarwinTOD: LLM Driven Lifelong Self Evolution for Task Oriented Dialog Systems

TL;DR

DarwinTOD addresses the static nature of deployed task-oriented dialog systems by introducing lifelong autonomous self-evolution. It couples an Evolvable Strategy Bank with a dual-loop architecture that combines online multi-agent dialog execution and offline evolutionary learning. The framework is formalized as a POMDP for dialog management and a Markov-chain population model for strategy evolution, with Boltzmann selection, Genesis/Mutation/Consolidation/Pruning operators, and a structured critique mechanism. Empirical results on MultiWOZ and SGD show state-of-the-art performance with sustained improvements, demonstrating that the system can bootstrap from minimal prior knowledge and autonomously specialize across domains while maintaining safety and interpretability.

Abstract

Traditional task-oriented dialog systems are unable to evolve from ongoing interactions or adapt to new domains after deployment, that is a critical limitation in real-world dynamic environments. Continual learning approaches depend on episodic retraining with human curated data, failing to achieve autonomy lifelong improvement. While evolutionary computation and LLM driven self improvement offer promising mechanisms for dialog optimization, they lack a unified framework for holistic, iterative strategy refinement. To bridge this gap, we propose DarwinTOD, a lifelong self evolving dialog framework that systematically integrates these two paradigms, enabling continuous strategy optimization from a zero-shot base without task specific fine-tuning. DarwinTOD maintains an Evolvable Strategy Bank and operates through a dual-loop process: online multi-agent dialog execution with peer critique, and offline structured evolutionary operations that refine the strategy bank using accumulated feedback. This closed-loop design enables autonomous continuous improvement without human intervention. Extensive experiments show that DarwinTOD surpasses previous state-of-the-art methods and exhibits continuous performance gains throughout evolution. Our work provides a novel framework for building dialog systems with lifelong self evolution capabilities.

Figures (9)

• Figure 1: Motivation comparison of TOD architectures. Both pipeline and end-to-end TOD systems suffer from cascaded errors or lack experience driven improvement, while DarwinTOD enables lifelong self evolution via a dual-loop architecture to achieve autonomous improvement.
• Figure 2: DarwinTOD's dual-loop algorithm framework. The online phase executes dialogs via multi-agent collaboration (DST/DP/NLG/UserSim) with peer critique, retrieving strategies from ESB through Boltzmann selection and logging interactions to SSM. The offline phase triggers evolutionary operations (Generate/Mutate/Consolidate/Prune) based on SSM feedback to update ESB, forming a closed loop for autonomous strategy refinement.
• Figure 3: Combine metric evolution across generations on MultiWOZ 2.0. All backbones show monotonic improvement, and the rapid early gains reflect the exploration-exploitation trade-off inherent in evolutionary optimization.
• Figure 4: Evolutionary dynamics of ESB across generations on MultiWOZ 2.0 with Qwen3-8B. The simultaneous rise and subsequent decline of entropy and fitness, coupled with increasing pairwise similarity, demonstrates a self organizing transition from exploratory diversity to exploitative convergence.
• Figure 5: t-SNE visualization of DP strategy embeddings: initial population.