CATCH: A Novel Data Synthesis Framework for High Therapy Fidelity and Memory-Driven Planning Chain of Thought in AI Counseling

TL;DR

CATCH tackles the problem of low therapy fidelity and opaque decision-making in AI counseling data by introducing a two-tier data synthesis framework. The Progressive Dialogue Synthesis (PDS) builds stage-aligned dialogues from client self-reports and personality traits, while Memory-Driven Dynamic Planning (MDP) CoT attaches explicit reasoning to each dialogue turn through a collaborative multi-agent pipeline. Empirical results show improvements in therapy-specific skills and general counseling competencies, with ablations confirming the value of both PDS and MDP CoT. The framework offers a scalable path toward higher-fidelity, more interpretable AI counseling, potentially reducing therapy drift and expanding access to effective support.

Abstract

Recently, advancements in AI counseling based on large language models have shown significant progress. However, existing studies employ a one-time generation approach to synthesize multi-turn dialogue samples, resulting in low therapy fidelity and failing to capture the decision-making rationale behind each response. In this work, we propose CATCH, a novel data synthesis framework designed to address these challenges. Specifically, to improve therapy fidelity, we introduce the Progressive Dialogue Synthesis strategy, which extracts goals, resources, and solutions from a client's self-report, organizes them into structured outlines, and then incrementally generates stage-aligned counseling dialogues. To capture decision-making rationale behind each response, we propose the Memory-Driven Dynamic Planning thinking pattern that integrates memory enhancement, global planning, and strategy reasoning; a collaborative multi-agent optimizer then leverages MDP to attach explicit chain-of-thought to each dialogue turn. Extensive experiments and human evaluations demonstrate that CATCH significantly enhances fidelity and logical coherence in AI counseling.

Figures (8)

• Figure 1: The Progressive Dialogue Synthesis (PDS) strategy for high-fidelity dialogue generation. PDS translates therapeutic principles into a structured process: from client self-reports and personality traits, it derives goals, resources, and solutions, forming a stage-guided outline that ensures fidelity in the incremental generation of counseling dialogues.
• Figure 2: The Memory-Driven Dynamic Planning (MDP) Chain-of-Thought (CoT) synthesis pipeline. The framework employs a collaborative multi-agent system to generate structured counseling reasoning: the Memory Capture Agent summarizes and extracts key information from the dialogue history, the Global Plan Agent determines therapeutic stage and progression, and the Strategy Reasoning Agent refines response strategies. Each step is verified by a Checking Agent for consistency, and the Fusion Agent integrates validated components into a coherent, first-person thought process.
• Figure 3: Comparison of expert evaluations between dialogues generated by the PDS strategy and the one-time generation approach. Detailed definitions of each metric are provided in Table \ref{['appendix:metrics']}.
• Figure 4: Performance comparison across different client attitudes. The bar chart shows the average evaluation scores of various models under positive, neutral, and negative client attitude settings. The dotted line indicates the overall average score across all attitudes. Notably, baseline models exhibit a pronounced performance gap, particularly struggling with negative-attitude clients. This highlights a critical weakness in real-world counseling scenarios.
• Figure 5: PDS dialogue sample page 1 of 3.