Trust Modeling in Counseling Conversations: A Benchmark Study

TL;DR

This work tackles modeling trust dynamics in counseling dialogues by defining trust as a dynamic trajectory and introducing MENTAL-TRUST, a 212-session dataset with seven expert-verified ordinal trust levels. It establishes TRUST-BENCH to benchmark diverse language models on trust detection as an ordinal classification task and analyzes trust trajectories and topic alignment across outcomes. The findings reveal that smaller, specialized models (e.g., Mental-BART, DeBERTa) often outperform large LLMs in capturing nuanced trust fluctuations, with clear patterns in trust evolution (refusal → digression → opening-up) and topic-concordant outcomes. The study provides a foundation for real-time therapeutic recalibration and highlights directions for expanding trustworthy, ethically designed AI support in mental health counseling.

Abstract

In mental health counseling, a variety of earlier studies have focused on dialogue modeling. However, most of these studies give limited to no emphasis on the quality of interaction between a patient and a therapist. The therapeutic bond between a patient and a therapist directly correlates with effective mental health counseling. It involves developing the patient's trust on the therapist over the course of counseling. To assess the therapeutic bond in counseling, we introduce trust as a therapist-assistive metric. Our definition of trust involves patients' willingness and openness to express themselves and, consequently, receive better care. We conceptualize it as a dynamic trajectory observable through textual interactions during the counseling. To facilitate trust modeling, we present MENTAL-TRUST, a novel counseling dataset comprising manual annotation of 212 counseling sessions with first-of-its-kind seven expert-verified ordinal trust levels. We project our problem statement as an ordinal classification task for trust quantification and propose a new benchmark, TrustBench, comprising a suite of classical and state-of-the-art language models on MENTAL-TRUST. We evaluate the performance across a suite of metrics and lay out an exhaustive set of findings. Our study aims to unfold how trust evolves in therapeutic interactions.

Figures (4)

• Figure 1: Distribution of ridge density for each trust level between 1 (min) -- 4 (max) in MENTAL-TRUST.
• Figure 2: Examples of trust progression scenarios (increasing, constant, decreasing) across top-performing models. The ground-truth trust values are represented by the blue line. Decoder-only models, particularly domain-specific variants such as Mental-BART, achieve the closest alignment with true trust values, while closed-source models perform the worst due to their inherent rigidity.
• Figure 3: Topic of discussion in counseling sessions segmented into two major segments. Top: illustrating core topics in sessions with a positive outcome; Bottom: showing topics in sessions with negative outcomes.
• Figure 4: Illustration of the progression in trust scores, represented as either increasing or decreasing jumps. The y-axis denotes the starting trust score ("From"), while the x-axis indicates the resulting trust score ("To"). For instance, a progression from 2.0 to 2.5 corresponds to the (4, 5) cell in the upper triangular tables. Increasing trust score jumps are shown in green, while decreasing jumps are displayed in orange. Darker shades signify higher counts of the respective trust score changes.