Mitigating Unhelpfulness in Emotional Support Conversations with Multifaceted AI Feedback

TL;DR

This work tackles the problem of unhelpful responses in emotional support conversations by introducing Muffin, a model-agnostic framework that integrates a Multifaceted AI Feedback module and a contrastive learning objective. The feedback module assesses unhelpfulness across three emotional-support facets (empathy, skill efficiency, coherence) using instruction-tuned LLaMA with LoRA, enabling automated, facet-aware labeling. Training employs a contrastive loss alongside the standard generation objective to steer the model away from unhelpful outputs, and is validated across five state-of-the-art base models on the ESConv dataset, showing improvements in both automatic metrics and human judgments. The approach enhances perceived helpfulness and practical usefulness of responses while highlighting considerations around facet interactions, dataset construction, and ethical deployment in sensitive support contexts.

Abstract

An emotional support conversation system aims to alleviate users' emotional distress and assist them in addressing their challenges. To generate supportive responses, it is critical to consider multiple factors such as empathy, support strategies, and response coherence, as established in prior methods. Nonetheless, previous models occasionally generate unhelpful responses, which intend to provide support but display counterproductive effects. According to psychology and communication theories, poor performance in just one contributing factor might cause a response to be unhelpful. From the model training perspective, since these models have not been exposed to unhelpful responses during their training phase, they are unable to distinguish if the tokens they generate might result in unhelpful responses during inference. To address this issue, we introduce a novel model-agnostic framework named mitigating unhelpfulness with multifaceted AI feedback for emotional support (Muffin). Specifically, Muffin employs a multifaceted AI feedback module to assess the helpfulness of responses generated by a specific model with consideration of multiple factors. Using contrastive learning, it then reduces the likelihood of the model generating unhelpful responses compared to the helpful ones. Experimental results demonstrate that Muffin effectively mitigates the generation of unhelpful responses while slightly increasing response fluency and relevance.

Figures (9)

• Figure 1: Examples of unhelpful responses generated by recent emotional support conversation models, including BlenderBot roller-etal-2021-recipes, MultiESC cheng-etal-2022-improving, and KEMI deng2023knowledge.
• Figure 2: The overview of our proposed model-agnostic framework— Muffin. $\fcolorbox{line+}{box+}{$+$}$ and $\fcolorbox{line-}{box-}{$-$}$ indicate helpful (non-unhelpful) and unhelpful labels, respectively.
• Figure 3: The prompt used by the Multifaceted AI Feedback for classifying the supporter's response.
• Figure 4: Comparison of performance among various Language Model Models (LLMs) including GPT-3.5, GPT-4, LLaMA (Vanilla), and LLaMA (Tuned) in classification tasks related to different facets of emotional support, as well as the aggregated feedback.
• Figure 5: Human A/B test results. Displayed within each bar, from left to right, are the ratios for "Muffin Wins", "Tie", and "Base Wins".