Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Scaffolding Empathy: Training Counselors with Simulated Patients and Utterance-level Performance Visualizations

Ian Steenstra, Farnaz Nouraei, Timothy W. Bickmore

TL;DR

Motivational interviewing training traditionally relies on intermittent feedback from role-plays with standardized patients, limiting skill acquisition. The authors introduce SimPatient, an LLM-powered simulated patient with an integrated, utterance-level performance visualization dashboard, driven by a multi-agent architecture that also delivers rationale-backed feedback and dynamic models of patient cognition. A formative study guided the design around MI metrics, visualization modalities, and transparency, followed by a MI training study demonstrating high usability and increased MI self-efficacy among both professional and student counselors. The work suggests that automated, explainable feedback and cognitive-state visualizations can accelerate counselor training and generalize to other social-skills domains, while highlighting challenges in simulating resistance and persona fidelity that warrant further research.

Abstract

Learning therapeutic counseling involves significant role-play experience with mock patients, with current manual training methods providing only intermittent granular feedback. We seek to accelerate and optimize counselor training by providing frequent, detailed feedback to trainees as they interact with a simulated patient. Our first application domain involves training motivational interviewing skills for counselors. Motivational interviewing is a collaborative counseling style in which patients are guided to talk about changing their behavior, with empathetic counseling an essential ingredient. We developed and evaluated an LLM-powered training system that features a simulated patient and visualizations of turn-by-turn performance feedback tailored to the needs of counselors learning motivational interviewing. We conducted an evaluation study with professional and student counselors, demonstrating high usability and satisfaction with the system. We present design implications for the development of automated systems that train users in counseling skills and their generalizability to other types of social skills training.

Scaffolding Empathy: Training Counselors with Simulated Patients and Utterance-level Performance Visualizations

TL;DR

Motivational interviewing training traditionally relies on intermittent feedback from role-plays with standardized patients, limiting skill acquisition. The authors introduce SimPatient, an LLM-powered simulated patient with an integrated, utterance-level performance visualization dashboard, driven by a multi-agent architecture that also delivers rationale-backed feedback and dynamic models of patient cognition. A formative study guided the design around MI metrics, visualization modalities, and transparency, followed by a MI training study demonstrating high usability and increased MI self-efficacy among both professional and student counselors. The work suggests that automated, explainable feedback and cognitive-state visualizations can accelerate counselor training and generalize to other social-skills domains, while highlighting challenges in simulating resistance and persona fidelity that warrant further research.

Abstract

Learning therapeutic counseling involves significant role-play experience with mock patients, with current manual training methods providing only intermittent granular feedback. We seek to accelerate and optimize counselor training by providing frequent, detailed feedback to trainees as they interact with a simulated patient. Our first application domain involves training motivational interviewing skills for counselors. Motivational interviewing is a collaborative counseling style in which patients are guided to talk about changing their behavior, with empathetic counseling an essential ingredient. We developed and evaluated an LLM-powered training system that features a simulated patient and visualizations of turn-by-turn performance feedback tailored to the needs of counselors learning motivational interviewing. We conducted an evaluation study with professional and student counselors, demonstrating high usability and satisfaction with the system. We present design implications for the development of automated systems that train users in counseling skills and their generalizability to other types of social skills training.

Paper Structure

This paper contains 43 sections, 10 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Intelligent Systems for Clinical Skill Training
  4. Training Systems for Counseling Conversations
  5. Standardized Patients and Patient Simulation Systems
  6. LLMs for Simulated Patient Dialogue
  7. LLMs for Motivational Interviewing Evaluation
  8. LLM-Based Multi-Agent Systems
  9. Formative Study
  10. Study Procedure
  11. Results
  12. Participants
  13. Qualitative Results
  14. Design Outcomes
  15. SimPatient Design
  16. ...and 28 more sections

Figures (10)

  • Figure 1: SimPatient Design
  • Figure 2: Animated Character Model Variations
  • Figure 3: Graphical Evaluation Dashboard Examples: This figure showcases example graphs from our MI skills evaluation dashboard. a) a radar chart visually represents scores on key Global MI measures ("Partnership", "Empathy", "Cultivating Change Talk", "Softening Sustain Talk"), rated on a scale of 1 to 5. b) a bar graph displaying the frequency of specific MI behavior codes used during a session. c) a pie chart that depicts the percentage of MI-adherent and non-adherent behaviors, highlighting adherence to MI principles. d) an example of one of four proficiency comparison bar graphs, such as Percentage of Complex Reflections, that depict "Fair" and "Good" proficiency thresholds. Additional details of the evaluation dashboard are available in Appendix \ref{['apx:dashboard_example']}.
  • Figure 4: Dynamic Cognitive Factors Graph
  • Figure 5: A flowchart of the Motivational Interviewing Training Study
  • ...and 5 more figures