Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Pearl: A Review-driven Persona-Knowledge Grounded Conversational Recommendation Dataset

Minjin Kim, Minju Kim, Hana Kim, Beong-woo Kwak, Soyeon Chun, Hyunseo Kim, SeongKu Kang, Youngjae Yu, Jinyoung Yeo, Dongha Lee

TL;DR

This work presents a novel conversational recommendation dataset named PEARL, synthesized with persona- and knowledge-augmented LLM simulators, and demonstrates that utterances in PEARL include more specific user preferences, show expertise in the target domain, and provide recommendations more relevant to the dialogue context than those in prior datasets.

Abstract

Conversational recommender system is an emerging area that has garnered an increasing interest in the community, especially with the advancements in large language models (LLMs) that enable diverse reasoning over conversational input. Despite the progress, the field has many aspects left to explore. The currently available public datasets for conversational recommendation lack specific user preferences and explanations for recommendations, hindering high-quality recommendations. To address such challenges, we present a novel conversational recommendation dataset named PEARL, synthesized with persona- and knowledge-augmented LLM simulators. We obtain detailed persona and knowledge from real-world reviews and construct a large-scale dataset with over 57k dialogues. Our experimental results demonstrate that utterances in PEARL include more specific user preferences, show expertise in the target domain, and provide recommendations more relevant to the dialogue context than those in prior datasets.

Pearl: A Review-driven Persona-Knowledge Grounded Conversational Recommendation Dataset

TL;DR

This work presents a novel conversational recommendation dataset named PEARL, synthesized with persona- and knowledge-augmented LLM simulators, and demonstrates that utterances in PEARL include more specific user preferences, show expertise in the target domain, and provide recommendations more relevant to the dialogue context than those in prior datasets.

Abstract

Conversational recommender system is an emerging area that has garnered an increasing interest in the community, especially with the advancements in large language models (LLMs) that enable diverse reasoning over conversational input. Despite the progress, the field has many aspects left to explore. The currently available public datasets for conversational recommendation lack specific user preferences and explanations for recommendations, hindering high-quality recommendations. To address such challenges, we present a novel conversational recommendation dataset named PEARL, synthesized with persona- and knowledge-augmented LLM simulators. We obtain detailed persona and knowledge from real-world reviews and construct a large-scale dataset with over 57k dialogues. Our experimental results demonstrate that utterances in PEARL include more specific user preferences, show expertise in the target domain, and provide recommendations more relevant to the dialogue context than those in prior datasets.
Paper Structure (41 sections, 8 figures, 17 tables)

This paper contains 41 sections, 8 figures, 17 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Conversational Recommendation
  4. Synthetic Data Generation
  5. Pearl Construction
  6. Constructing User-Review and Item-Review Databases
  7. Persona-augmented User Simulator
  8. Persona components.
  9. Utterance generation.
  10. Knowledge-augmented Recommender Simulator
  11. Item retrieval for recommendation.
  12. Item recommendation with reasoning.
  13. Dialogue Generation
  14. Dialogue Filtering
  15. Basic filtering.
  16. ...and 26 more sections

Figures (8)

  • Figure 1: An example comparing utterances of crowdworkers and our persona-knowledge augmented LLM simulators.
  • Figure 2: The overview of Pearl construction method. We synthesize recommendation dialogues with review-driven persona-knowledge grounded simulators. Specifically, our user simulator is equipped with persona and our recommender simulator is augmented by knowledge derived from reviews.
  • Figure 3: Results of human evaluation on head-to-head comparison between conversations sampled from Pearl and those from ReDial. (*: p-value < 0.05)
  • Figure 4: Results of human evaluation on head-to-head comparison between conversations sampled from Pearl and those from INSPIRED. (*: p-value < 0.05)
  • Figure 5: Results of head-to-head comparison human evaluation between responses generated from BART trained on Pearl and on ReDial. (*: p-value < 0.05)
  • ...and 3 more figures