PCQPR: Proactive Conversational Question Planning with Reflection

TL;DR

This work redefine the CQG task as Conclusion-driven Conversational Question Generation (CCQG) by focusing on proactivity, not merely reacting to the unfolding conversation but actively steering it towards a conclusion-oriented question-answer pair.

Abstract

Conversational Question Generation (CQG) enhances the interactivity of conversational question-answering systems in fields such as education, customer service, and entertainment. However, traditional CQG, focusing primarily on the immediate context, lacks the conversational foresight necessary to guide conversations toward specified conclusions. This limitation significantly restricts their ability to achieve conclusion-oriented conversational outcomes. In this work, we redefine the CQG task as Conclusion-driven Conversational Question Generation (CCQG) by focusing on proactivity, not merely reacting to the unfolding conversation but actively steering it towards a conclusion-oriented question-answer pair. To address this, we propose a novel approach, called Proactive Conversational Question Planning with self-Refining (PCQPR). Concretely, by integrating a planning algorithm inspired by Monte Carlo Tree Search (MCTS) with the analytical capabilities of large language models (LLMs), PCQPR predicts future conversation turns and continuously refines its questioning strategies. This iterative self-refining mechanism ensures the generation of contextually relevant questions strategically devised to reach a specified outcome. Our extensive evaluations demonstrate that PCQPR significantly surpasses existing CQG methods, marking a paradigm shift towards conclusion-oriented conversational question-answering systems.

Figures (8)

• Figure 1: An example of CCQG. Given the specific context, the initial conversation (blue), and the concluding question-answer pair (orange), CCQG proactively generates subsequent questions (green), aiming to advance toward the predefined question-answer pair.
• Figure 2: An overview of our PCQPR framework. Specifically, the framework employs MCTS-like for planning, performing a lookahead search to generate the desired response (marked in black). Subsequently, it leverages a comparable reflection strategy to elevate the quality of this response (marked in red).
• Figure 3: The prompt for response generation.
• Figure 4: The prompt for reflection generation.
• Figure 5: Sensitivity study.