Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

PsychAgent: An Experience-Driven Lifelong Learning Agent for Self-Evolving Psychological Counselor

Yutao Yang, Junsong Li, Qianjun Pan, Jie Zhou, Kai Chen, Qin Chen, Jingyuan Zhao, Ningning Zhou, Xin Li, Liang He

Abstract

Existing methods for AI psychological counselors predominantly rely on supervised fine-tuning using static dialogue datasets. However, this contrasts with human experts, who continuously refine their proficiency through clinical practice and accumulated experience. To bridge this gap, we propose an Experience-Driven Lifelong Learning Agent (\texttt{PsychAgent}) for psychological counseling. First, we establish a Memory-Augmented Planning Engine tailored for longitudinal multi-session interactions, which ensures therapeutic continuity through persistent memory and strategic planning. Second, to support self-evolution, we design a Skill Evolution Engine that extracts new practice-grounded skills from historical counseling trajectories. Finally, we introduce a Reinforced Internalization Engine that integrates the evolved skills into the model via rejection fine-tuning, aiming to improve performance across diverse scenarios. Comparative analysis shows that our approach achieves higher scores than strong general LLMs (e.g., GPT-5.4, Gemini-3) and domain-specific baselines across all reported evaluation dimensions. These results suggest that lifelong learning can improve the consistency and overall quality of multi-session counseling responses.

PsychAgent: An Experience-Driven Lifelong Learning Agent for Self-Evolving Psychological Counselor

Abstract

Existing methods for AI psychological counselors predominantly rely on supervised fine-tuning using static dialogue datasets. However, this contrasts with human experts, who continuously refine their proficiency through clinical practice and accumulated experience. To bridge this gap, we propose an Experience-Driven Lifelong Learning Agent (\texttt{PsychAgent}) for psychological counseling. First, we establish a Memory-Augmented Planning Engine tailored for longitudinal multi-session interactions, which ensures therapeutic continuity through persistent memory and strategic planning. Second, to support self-evolution, we design a Skill Evolution Engine that extracts new practice-grounded skills from historical counseling trajectories. Finally, we introduce a Reinforced Internalization Engine that integrates the evolved skills into the model via rejection fine-tuning, aiming to improve performance across diverse scenarios. Comparative analysis shows that our approach achieves higher scores than strong general LLMs (e.g., GPT-5.4, Gemini-3) and domain-specific baselines across all reported evaluation dimensions. These results suggest that lifelong learning can improve the consistency and overall quality of multi-session counseling responses.

Paper Structure

This paper contains 37 sections, 3 equations, 3 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. LLMs for Psychological Counseling
  4. Experience-Driven Lifelong Learning
  5. Our Methods
  6. Memory-Augmented Planning Engine
  7. Dynamic Memory Construction.
  8. Goal-Oriented Strategic Planning.
  9. Skill Evolution Engine
  10. Hierarchical Skill Taxonomy.
  11. Context-Aware Skill Retrieval.
  12. Generative Skill Evolution.
  13. Reinforced Internalization Engine
  14. Iterative Trajectory Rollout and Selection.
  15. Internalization via Rejection Fine-Tuning.
  16. ...and 22 more sections

Figures (3)

  • Figure 1: The overall architecture of PsychAgent, an experience-driven lifelong learning framework for psychological counseling. The system operates through a synergistic closed-loop mechanism: (a) The Memory-Augmented Planning Engine ensures therapeutic continuity across longitudinal sessions by reasoning over dynamic client profiles and formulating strategic goals; (b) The Skill Evolution Engine manages a hierarchical repository of therapeutic techniques, supporting both real-time context-aware retrieval and the post-session abstraction of novel skills; (c) The Reinforced Internalization Engine solidifies these capabilities via rejection fine-tuning, where optimal trajectories are selected to transform explicit skills into the model's endogenous intuition.
  • Figure 2: Clients' Emotional Trajectories.
  • Figure 3: Detailed results across Counselor- and Client-Level therapy metrics