IMBUE: Improving Interpersonal Effectiveness through Simulation and Just-in-time Feedback with Human-Language Model Interaction

TL;DR

IMBUE presents a human–LM interactive training system that couples LM-based simulation of challenging conversations with expert-informed just-in-time feedback grounded in the DBT DEAR MAN framework. By building an expert-annotated dataset and employing contras tive prompting with kNN demonstrations, the approach yields feedback that closely aligns with clinical expertise. A randomized trial demonstrates that adding just-in-time feedback substantially improves skill mastery, self-efficacy, and emotion regulation beyond simulation alone, though generalization to new emotional contexts is limited without additional practice. The work advances accessible, psychology-grounded interpersonal skills training and illustrates how human-LM collaboration can support nuanced, emotion-aware communication learning.

Abstract

Navigating certain communication situations can be challenging due to individuals' lack of skills and the interference of strong emotions. However, effective learning opportunities are rarely accessible. In this work, we conduct a human-centered study that uses language models to simulate bespoke communication training and provide just-in-time feedback to support the practice and learning of interpersonal effectiveness skills. We apply the interpersonal effectiveness framework from Dialectical Behavioral Therapy (DBT), DEAR MAN, which focuses on both conversational and emotional skills. We present IMBUE, an interactive training system that provides feedback 25% more similar to experts' feedback, compared to that generated by GPT-4. IMBUE is the first to focus on communication skills and emotion management simultaneously, incorporate experts' domain knowledge in providing feedback, and be grounded in psychology theory. Through a randomized trial of 86 participants, we find that IMBUE's simulation-only variant significantly improves participants' self-efficacy (up to 17%) and reduces negative emotions (up to 25%). With IMBUE's additional just-in-time feedback, participants demonstrate 17% improvement in skill mastery, along with greater enhancements in self-efficacy (27% more) and reduction of negative emotions (16% more) compared to simulation-only. The improvement in skill mastery is the only measure that is transferred to new and more difficult situations; situation specific training is necessary for improving self-efficacy and emotion reduction.

Figures (13)

• Figure 1: Overview of Imbue, an interactive training system that (A) simulates bespoke communication situations and (B) provides expert-like just-in-time feedback based on (C) the DEAR MAN framework. Imbue is backed by LMs that perform two tasks: (a) Next skill suggestion: before a user writes a message, Imbue suggests a skill to apply (§\ref{['subsec:method-suggest-skill']}). (b) Feedback on skill use: after a user writes a message, Imbue provides skill rating and improvement suggestions (§\ref{['subsec:method-skill-use']}).
• Figure 2: User study experimental design. We randomly assigned participants to one of simulation-only and simulation+feedback groups. Each participant was asked to provide two situations, S1 and S2. Only S1 was used in training. Both S1 and S2 were used in pre- and post-training self-efficacy and emotion intensity surveys and in post-training skill-use evaluation through chat interaction.
• Figure 3: Improvement in skill mastery. Simulation+feedback group shows a significantly higher improvement in skill mastery (17.6% on a 0-2 scale, **, $d$=0.59) compared to simulation-only (0.1%) after only one training session. The difference is also significant for the subset of conversational skills that participants choose to use in each utterance (only measured when the skills are chosen), Describe, Express, Assert, Reinforce, and Negotiate (24.8%, **, $d$=0.59) and state-of-mind skills (measured in every utterance), Mindful and Confident (15.7%, **, $d$=0.59). (***: $p<.001$, **:$p<.01$, *:$p<.05$. $d$: Cohen's $d$.)
• Figure 4: Change of self-reported efficacy and emotional intensity for both Situation 1 (S1) and Situation 2 (S2) after a single training session on S1. Gray area indicates the direction of improvement for each score. The group receiving just-in-time feedback generated with our method in addition to conversation simulation see significant increase in their confidence (43.6%, ***), hopefulness (11.0%, *), motivation (22.1%, ***) towards having the conversation, significant decrease in their worrying thoughts (30.9%, ***) about having the conversation and their anger (23.5%, *), fear (40.9%, ***), and sadness (29.0%, ***) towards the training situation (S1). The increase in confidence and reduction in fear are 26.7% (**, $d$=0.57) and 15.7% (*,$d$=0.51) significantly more than the group receiving simulation only. This improvement in self-efficacy and emotional reduction does not transfer immediately to a new, more difficult situation (S2). See Section \ref{['sec:user_study']} for more analysis and discussion.
• Figure 5: Difference between Simulation+Feedback group and Simulation-only group on the improvement of skill use by each skill. We use bootstrapping to estimate confidence intervals (5000 iterations). Simulation+Feedback group sees a significantly higher increase in overall skill use (15.6%, $p=.000$), Express(43.2%, $p=.003$), Mindful(11.6%, $p=.012$), and Confident skills(10.8%, $p=.021$). ***: $p<.001$, **:$p<.01$, *:$p<.05$, $d$: Cohen's $d$.