Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Question Suggestion for Conversational Shopping Assistants Using Product Metadata

Nikhita Vedula, Oleg Rokhlenko, Shervin Malmasi

TL;DR

This work tackles the challenge of helping customers effectively interact with conversational shopping assistants by automatically generating product-grounded questions to guide conversations. It introduces an LLM-based framework that uses in-context learning and supervised fine-tuning to produce diverse, relevant, answerable, and fluent questions anchored in catalog metadata and reviews, with each question paired with its contextual answer. Extensive offline evaluation using GPT-4 and human judgments demonstrates solid performance on relevance and fluency, with room for improvement in usefulness, answerability, and stylistic aspects. The findings suggest that diverse, question-based prompts can reduce conversational friction, enable downstream applications like FAQs and RAG, and can be efficiently deployed with latency optimizations such as caching or streaming outputs.

Abstract

Digital assistants have become ubiquitous in e-commerce applications, following the recent advancements in Information Retrieval (IR), Natural Language Processing (NLP) and Generative Artificial Intelligence (AI). However, customers are often unsure or unaware of how to effectively converse with these assistants to meet their shopping needs. In this work, we emphasize the importance of providing customers a fast, easy to use, and natural way to interact with conversational shopping assistants. We propose a framework that employs Large Language Models (LLMs) to automatically generate contextual, useful, answerable, fluent and diverse questions about products, via in-context learning and supervised fine-tuning. Recommending these questions to customers as helpful suggestions or hints to both start and continue a conversation can result in a smoother and faster shopping experience with reduced conversation overhead and friction. We perform extensive offline evaluations, and discuss in detail about potential customer impact, and the type, length and latency of our generated product questions if incorporated into a real-world shopping assistant.

Question Suggestion for Conversational Shopping Assistants Using Product Metadata

TL;DR

This work tackles the challenge of helping customers effectively interact with conversational shopping assistants by automatically generating product-grounded questions to guide conversations. It introduces an LLM-based framework that uses in-context learning and supervised fine-tuning to produce diverse, relevant, answerable, and fluent questions anchored in catalog metadata and reviews, with each question paired with its contextual answer. Extensive offline evaluation using GPT-4 and human judgments demonstrates solid performance on relevance and fluency, with room for improvement in usefulness, answerability, and stylistic aspects. The findings suggest that diverse, question-based prompts can reduce conversational friction, enable downstream applications like FAQs and RAG, and can be efficiently deployed with latency optimizations such as caching or streaming outputs.

Abstract

Digital assistants have become ubiquitous in e-commerce applications, following the recent advancements in Information Retrieval (IR), Natural Language Processing (NLP) and Generative Artificial Intelligence (AI). However, customers are often unsure or unaware of how to effectively converse with these assistants to meet their shopping needs. In this work, we emphasize the importance of providing customers a fast, easy to use, and natural way to interact with conversational shopping assistants. We propose a framework that employs Large Language Models (LLMs) to automatically generate contextual, useful, answerable, fluent and diverse questions about products, via in-context learning and supervised fine-tuning. Recommending these questions to customers as helpful suggestions or hints to both start and continue a conversation can result in a smoother and faster shopping experience with reduced conversation overhead and friction. We perform extensive offline evaluations, and discuss in detail about potential customer impact, and the type, length and latency of our generated product questions if incorporated into a real-world shopping assistant.
Paper Structure (18 sections, 1 figure, 3 tables)

This paper contains 18 sections, 1 figure, 3 tables.

Table of Contents

  1. Introduction and Background
  2. Problem Statement and Dataset
  3. Our Approach
  4. Product Question Quality Criteria
  5. Product Question Suggestion Generation
  6. In-Context Learning:
  7. Supervised Fine-tuning:
  8. Experiments and Results
  9. Data:
  10. Models:
  11. Evaluation Metrics:
  12. Automatic Evaluation:
  13. Human Evaluation:
  14. Example Question Suggestions:
  15. Discussion
  16. ...and 3 more sections

Figures (1)

  • Figure 1: Prompt for ICL-based product question generation.