Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CPsyCoun: A Report-based Multi-turn Dialogue Reconstruction and Evaluation Framework for Chinese Psychological Counseling

Chenhao Zhang, Renhao Li, Minghuan Tan, Min Yang, Jingwei Zhu, Di Yang, Jiahao Zhao, Guancheng Ye, Chengming Li, Xiping Hu

TL;DR

CPsyCoun tackles the scarcity of authentic Chinese psychological counseling data and the lack of multi-turn evaluation by reconstructing dialogues from anonymized reports and establishing an automatic evaluation benchmark. It introduces a two-phase Memo2Demo reconstruction method, builds the CPsyCounD dataset (3,134 dialogues) and the CPsyCounX model trained on it, and deploys the CPsyCounE evaluation framework. Experimental results show substantial gains in dialogue quality and counselor professionalism/authenticity, demonstrating the framework’s effectiveness for AI-assisted psychological counseling. The work provides open-source datasets and models to propel research in LLM-assisted mental health applications.

Abstract

Using large language models (LLMs) to assist psychological counseling is a significant but challenging task at present. Attempts have been made on improving empathetic conversations or acting as effective assistants in the treatment with LLMs. However, the existing datasets lack consulting knowledge, resulting in LLMs lacking professional consulting competence. Moreover, how to automatically evaluate multi-turn dialogues within the counseling process remains an understudied area. To bridge the gap, we propose CPsyCoun, a report-based multi-turn dialogue reconstruction and evaluation framework for Chinese psychological counseling. To fully exploit psychological counseling reports, a two-phase approach is devised to construct high-quality dialogues while a comprehensive evaluation benchmark is developed for the effective automatic evaluation of multi-turn psychological consultations. Competitive experimental results demonstrate the effectiveness of our proposed framework in psychological counseling. We open-source the datasets and model for future research at https://github.com/CAS-SIAT-XinHai/CPsyCoun

CPsyCoun: A Report-based Multi-turn Dialogue Reconstruction and Evaluation Framework for Chinese Psychological Counseling

TL;DR

CPsyCoun tackles the scarcity of authentic Chinese psychological counseling data and the lack of multi-turn evaluation by reconstructing dialogues from anonymized reports and establishing an automatic evaluation benchmark. It introduces a two-phase Memo2Demo reconstruction method, builds the CPsyCounD dataset (3,134 dialogues) and the CPsyCounX model trained on it, and deploys the CPsyCounE evaluation framework. Experimental results show substantial gains in dialogue quality and counselor professionalism/authenticity, demonstrating the framework’s effectiveness for AI-assisted psychological counseling. The work provides open-source datasets and models to propel research in LLM-assisted mental health applications.

Abstract

Using large language models (LLMs) to assist psychological counseling is a significant but challenging task at present. Attempts have been made on improving empathetic conversations or acting as effective assistants in the treatment with LLMs. However, the existing datasets lack consulting knowledge, resulting in LLMs lacking professional consulting competence. Moreover, how to automatically evaluate multi-turn dialogues within the counseling process remains an understudied area. To bridge the gap, we propose CPsyCoun, a report-based multi-turn dialogue reconstruction and evaluation framework for Chinese psychological counseling. To fully exploit psychological counseling reports, a two-phase approach is devised to construct high-quality dialogues while a comprehensive evaluation benchmark is developed for the effective automatic evaluation of multi-turn psychological consultations. Competitive experimental results demonstrate the effectiveness of our proposed framework in psychological counseling. We open-source the datasets and model for future research at https://github.com/CAS-SIAT-XinHai/CPsyCoun
Paper Structure (30 sections, 2 equations, 11 figures, 5 tables)

This paper contains 30 sections, 2 equations, 11 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Dialogue Generation and Reconstruction using LLMs
  4. Evaluation of Generated Dialogues using LLMs
  5. CPsyCoun
  6. Data Collection
  7. Data Processing
  8. Data Format
  9. Data analysis
  10. Dialogue Generation Method for Psychological Counseling
  11. Baseline Method
  12. Memo2Demo
  13. Memo Conversion
  14. Demo Generation
  15. Automatic Evaluation of LLM-based Psychological Counseling
  16. ...and 15 more sections

Figures (11)

  • Figure 1: The general framework of CPsyCoun
  • Figure 2: Statistics of collected cases.
  • Figure 3: Illustration of the dialogue reconstruction method Memo2Demo
  • Figure 4: Radar plot of detailed scores of CPsyCounX and other baselines on 9 counseling topics on CPsyCounE.
  • Figure 5: Description of report format together with an example from our collection
  • ...and 6 more figures