Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CARE: An Explainable Computational Framework for Assessing Client-Perceived Therapeutic Alliance Using Large Language Models

Anqi Li, Chenxiao Wang, Yu Lu, Renjun Xu, Lizhi Ma, Zhenzhong Lan

TL;DR

Experiments show that CARE outperforms leading LLMs and substantially reduces the gap between counselor evaluations and client-perceived alliance, achieving over 70% higher Pearson correlation with client ratings, demonstrating its potential as an AI-assisted tool for supporting mental health care.

Abstract

Client perceptions of the therapeutic alliance are critical for counseling effectiveness. Accurately capturing these perceptions remains challenging, as traditional post-session questionnaires are burdensome and often delayed, while existing computational approaches produce coarse scores, lack interpretable rationales, and fail to model holistic session context. We present CARE, an LLM-based framework to automatically predict multi-dimensional alliance scores and generate interpretable rationales from counseling transcripts. Built on the CounselingWAI dataset and enriched with 9,516 expert-curated rationales, CARE is fine-tuned using rationale-augmented supervision with the LLaMA-3.1-8B-Instruct backbone. Experiments show that CARE outperforms leading LLMs and substantially reduces the gap between counselor evaluations and client-perceived alliance, achieving over 70% higher Pearson correlation with client ratings. Rationale-augmented supervision further improves predictive accuracy. CARE also produces high-quality, contextually grounded rationales, validated by both automatic and human evaluations. Applied to real-world Chinese online counseling sessions, CARE uncovers common alliance-building challenges, illustrates how interaction patterns shape alliance development, and provides actionable insights, demonstrating its potential as an AI-assisted tool for supporting mental health care.

CARE: An Explainable Computational Framework for Assessing Client-Perceived Therapeutic Alliance Using Large Language Models

TL;DR

Experiments show that CARE outperforms leading LLMs and substantially reduces the gap between counselor evaluations and client-perceived alliance, achieving over 70% higher Pearson correlation with client ratings, demonstrating its potential as an AI-assisted tool for supporting mental health care.

Abstract

Client perceptions of the therapeutic alliance are critical for counseling effectiveness. Accurately capturing these perceptions remains challenging, as traditional post-session questionnaires are burdensome and often delayed, while existing computational approaches produce coarse scores, lack interpretable rationales, and fail to model holistic session context. We present CARE, an LLM-based framework to automatically predict multi-dimensional alliance scores and generate interpretable rationales from counseling transcripts. Built on the CounselingWAI dataset and enriched with 9,516 expert-curated rationales, CARE is fine-tuned using rationale-augmented supervision with the LLaMA-3.1-8B-Instruct backbone. Experiments show that CARE outperforms leading LLMs and substantially reduces the gap between counselor evaluations and client-perceived alliance, achieving over 70% higher Pearson correlation with client ratings. Rationale-augmented supervision further improves predictive accuracy. CARE also produces high-quality, contextually grounded rationales, validated by both automatic and human evaluations. Applied to real-world Chinese online counseling sessions, CARE uncovers common alliance-building challenges, illustrates how interaction patterns shape alliance development, and provides actionable insights, demonstrating its potential as an AI-assisted tool for supporting mental health care.
Paper Structure (46 sections, 4 figures, 9 tables)

This paper contains 46 sections, 4 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Computational Alliance Assessment.
  4. LLMs in Mental Health Conversation Analysis.
  5. Measurement of Therapeutic Alliance
  6. Data Collection
  7. Data Source
  8. Rationale Augmentation
  9. Data Quality
  10. Privacy and Ethics
  11. CARE Model
  12. Task Definition
  13. Data Preparation
  14. Experimental Setup
  15. Baselines
  16. ...and 31 more sections

Figures (4)

  • Figure 1: The CARE model predicts client-perceived fine-grained working alliance scores after counseling sessions by identifying and summarizing explanatory reasons from the dialogue, compared to the traditional method where clients only provide scores through questionnaires.
  • Figure 2: Working alliance scores by dimension for the three most active counselors; dashed lines indicate overall counselor averages.
  • Figure 3: Distributions of client ratings across the three working alliance dimensions—Goal, Task, and Bond—illustrated using histograms, box plots, and half-violin plots. Within each box, the X denotes the mean. The central line represents the median, the box corresponds to the interquartile range (IQR, 25th–75th percentile), and the whiskers extend to the full range of the data.
  • Figure 4: The template prompt for instructing LLMs to predict clients' perceived working alliance, using an example conversation and questionnaire question (displayed in cadetblue italic text).