DecoupledESC: Enhancing Emotional Support Generation via Strategy-Response Decoupled Preference Optimization

TL;DR

This work tackles persistent psychological errors and bias in Emotional Support Conversation models trained with supervised fine-tuning. It introduces Inferential Preference Mining (IPM) to create high-quality, disentangled preference data (IPM-PrefDial) and a Decoupled ESC framework that splits the task into Strategy Planning and Response Generation, optimized separately via Direct Preference Optimization. Empirical results show that the decoupled approach reduces preference bias and improves empathy, professionalism, fluency, and helpfulness, with human evaluations supporting the gains. The approach leverages the Extended Process Model of Emotion Regulation to justify the two-stage design and demonstrates practical improvements over joint optimization baselines. This decoupled methodology has potential to enhance scalable ESC systems and can be extended to larger models and multimodal settings.

Abstract

Recent advances in Emotional Support Conversation (ESC) have improved emotional support generation by fine-tuning Large Language Models (LLMs) via Supervised Fine-Tuning (SFT). However, common psychological errors still persist. While Direct Preference Optimization (DPO) shows promise in reducing such errors through pairwise preference learning, its effectiveness in ESC tasks is limited by two key challenges: (1) Entangled data structure: Existing ESC data inherently entangles psychological strategies and response content, making it difficult to construct high-quality preference pairs; and (2) Optimization ambiguity: Applying vanilla DPO to such entangled pairwise data leads to ambiguous training objectives. To address these issues, we introduce Inferential Preference Mining (IPM) to construct high-quality preference data, forming the IPM-PrefDial dataset. Building upon this data, we propose a Decoupled ESC framework inspired by Gross's Extended Process Model of Emotion Regulation, which decomposes the ESC task into two sequential subtasks: strategy planning and empathic response generation. Each was trained via SFT and subsequently enhanced by DPO to align with the psychological preference. Extensive experiments demonstrate that our Decoupled ESC framework outperforms joint optimization baselines, reducing preference bias and improving response quality.

Figures (20)

• Figure 1: Comparison from Vanilla-SFT to Vanilla-DPO to Decoupled-DPO. Vanilla-SFT lacks negative preference data, leading to high preference bias; Vanilla-DPO uses coupled preference data, causing potential negative optimization (regards PsNr, NsPr as pure negative samples); Decoupled-DPO decouples strategy and response, effectively reducing bias and psychological errors.
• Figure 2: Comparison of common psychological error type proportions among the FailedESConv dataset, Qwen-SFT inference results, and Llama-SFT inference results. Other Error refers to non-psychological errors.
• Figure 3: Strategy Distribution across different models.
• Figure 4: (a) Preference Bias and (b) Strategy Preference across Qwen and Llama models trained on different preference datasets.
• Figure 5: Comparison between previous vanilla SFT training paradigm and our proposed Decoupled ESC framework. The Decoupled ESC first undergoes SFT initialization, followed by DPO training using the IPM-PrefDial dataset.