Enhancing User-Oriented Proactivity in Open-Domain Dialogues with Critic Guidance

TL;DR

The paper addresses the lack of user-oriented proactivity in open-domain dialogues by introducing the User-oriented Proactive Chatbot (UPC). UPC leverages a learned critic (LLM-as-a-judge) to evaluate proactivity, uses critic-guided dialogue to generate high-quality corpora with diverse user backgrounds from the ISCO-800 dataset, and employs an iterative curriculum learning framework to progressively handle users with varying communication difficulties. Key contributions include the design of the proactivity critic, the critic-guided corpus generation paradigm, and the ISCO-800 dataset, along with empirical validation showing improved user-oriented relevance, interest, and value across multiple large language models. The approach demonstrates stronger engagement and attractiveness in open-domain conversations, suggesting practical benefits for human-computer interaction in social robotics and personal assistants.

Abstract

Open-domain dialogue systems aim to generate natural and engaging conversations, providing significant practical value in real applications such as social robotics and personal assistants. The advent of large language models (LLMs) has greatly advanced this field by improving context understanding and conversational fluency. However, existing LLM-based dialogue systems often fall short in proactively understanding the user's chatting preferences and guiding conversations toward user-centered topics. This lack of user-oriented proactivity can lead users to feel unappreciated, reducing their satisfaction and willingness to continue the conversation in human-computer interactions. To address this issue, we propose a User-oriented Proactive Chatbot (UPC) to enhance the user-oriented proactivity. Specifically, we first construct a critic to evaluate this proactivity inspired by the LLM-as-a-judge strategy. Given the scarcity of high-quality training data, we then employ the critic to guide dialogues between the chatbot and user agents, generating a corpus with enhanced user-oriented proactivity. To ensure the diversity of the user backgrounds, we introduce the ISCO-800, a diverse user background dataset for constructing user agents. Moreover, considering the communication difficulty varies among users, we propose an iterative curriculum learning method that trains the chatbot from easy-to-communicate users to more challenging ones, thereby gradually enhancing its performance. Experiments demonstrate that our proposed training method is applicable to different LLMs, improving user-oriented proactivity and attractiveness in open-domain dialogues.

Figures (7)

• Figure 1: Illustration of different open-domain dialogue methods. Left: universal LLMs. Medium: target-oriented methods. Right: our user-oriented method. The green dotted box means the intention of the chatbot.
• Figure 2: General diagram of proposed UPC. We train UPC in an iterative process. In each iteration, the chatbot engages in dialogue with user agents from the ISCO-800 dataset, guided by the critic to generate a corpus with enhanced proactivity. In the iterative curriculum learning process, we measure the communication difficulty and fine-tune the chatbot with the corpus corresponding to easy-to-communicate users. The fine-tuned chatbot is then used to generate the corpus for the next iteration. This iterative process repeats until convergence.
• Figure 3: The prompt of the critic.
• Figure 4: The regeneration prompt for the chatbot.
• Figure 5: Real user evaluation results between our UPC and Llama-3-70B-Instruct baseline.