Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Synthetic Dialogue Generation for Interactive Conversational Elicitation & Recommendation (ICER)

Moonkyung Ryu, Chih-Wei Hsu, Yinlam Chow, Mohammad Ghavamzadeh, Craig Boutilier

TL;DR

ICER tackles data scarcity in conversational recommender systems by integrating a behavior-grounded user simulator with LM-prompted dialogue refinement to generate realistic, multi-turn conversations. It introduces MD-DICER, a 100K MovieLens-based CRS dataset, and demonstrates that LM prompting yields more natural, consistent, and informative dialogues than templatized templates. Human and automated evaluations show that ICER-generated dialogues enhance user modeling and downstream LM-based recommendations, especially when preference elicitation and critiquing are involved. The work offers a reproducible methodology and dataset to advance behaviorally aware language-agent CRSs in data-scarce settings.

Abstract

While language models (LMs) offer great potential for conversational recommender systems (CRSs), the paucity of public CRS data makes fine-tuning LMs for CRSs challenging. In response, LMs as user simulators qua data generators can be used to train LM-based CRSs, but often lack behavioral consistency, generating utterance sequences inconsistent with those of any real user. To address this, we develop a methodology for generating natural dialogues that are consistent with a user's underlying state using behavior simulators together with LM-prompting. We illustrate our approach by generating a large, open-source CRS data set with both preference elicitation and example critiquing. Rater evaluation on some of these dialogues shows them to exhibit considerable consistency, factuality and naturalness.

Synthetic Dialogue Generation for Interactive Conversational Elicitation & Recommendation (ICER)

TL;DR

ICER tackles data scarcity in conversational recommender systems by integrating a behavior-grounded user simulator with LM-prompted dialogue refinement to generate realistic, multi-turn conversations. It introduces MD-DICER, a 100K MovieLens-based CRS dataset, and demonstrates that LM prompting yields more natural, consistent, and informative dialogues than templatized templates. Human and automated evaluations show that ICER-generated dialogues enhance user modeling and downstream LM-based recommendations, especially when preference elicitation and critiquing are involved. The work offers a reproducible methodology and dataset to advance behaviorally aware language-agent CRSs in data-scarce settings.

Abstract

While language models (LMs) offer great potential for conversational recommender systems (CRSs), the paucity of public CRS data makes fine-tuning LMs for CRSs challenging. In response, LMs as user simulators qua data generators can be used to train LM-based CRSs, but often lack behavioral consistency, generating utterance sequences inconsistent with those of any real user. To address this, we develop a methodology for generating natural dialogues that are consistent with a user's underlying state using behavior simulators together with LM-prompting. We illustrate our approach by generating a large, open-source CRS data set with both preference elicitation and example critiquing. Rater evaluation on some of these dialogues shows them to exhibit considerable consistency, factuality and naturalness.

Paper Structure

This paper contains 19 sections, 7 equations, 21 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. ICER: A Multi-turn User Simulator for Conversational Recommenders
  4. The Recommender Agent
  5. User Behavior Models
  6. Templatized Dialogue Generation
  7. LM-prompted Dialogue Generation
  8. MD-DICER: A MovieLens Dialogue Dataset using ICER
  9. Evaluation
  10. Rater Evaluation
  11. Automated Evaluation
  12. Conclusion
  13. Appendix
  14. ICER Framework Overview
  15. Example Simulation Trajectory
  16. ...and 4 more sections

Figures (21)

  • Figure 1: The ICER CRS Simulator Flowchart. The left part depicts decisions (represented with diamonds) and actions (represented with rectangles) in the recommender agent and the right part depicts decisions and actions in user behavior models.
  • Figure 2: An Example of Templatized Dialogue.
  • Figure 3: An Example of LM-prompted Dialogue given Fig. \ref{['fig:templ_dialog']}.
  • Figure 4: Prompt for Recommendation Agent Turn.
  • Figure 5: Prompt for Item Elicitation Agent Turn.
  • ...and 16 more figures