Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learn by Selling: Equipping Large Language Models with Product Knowledge for Context-Driven Recommendations

Sarthak Anand, Yutong Jiang, Giorgi Kokaia

TL;DR

The paper tackles enabling LLMs to deliver context-driven product recommendations by grounding them in a full product inventory. It introduces a data-generation pipeline that creates synthetic search queries embedding product IDs and generates corresponding sales responses, enabling supervised fine-tuning of an LLM. Using the Mistral7Bv0.2 model with product IDs as tokens, the approach improves ranking and category accuracy, but reveals significant challenges in factual accuracy and hallucination, especially for product details like series name and price. The work highlights practical implications for scalable, personalized recommendations and outlines directions to enhance adaptability to new products and reduce misinformation through targeted training and retrieval-based strategies.

Abstract

The rapid evolution of large language models (LLMs) has opened up new possibilities for applications such as context-driven product recommendations. However, the effectiveness of these models in this context is heavily reliant on their comprehensive understanding of the product inventory. This paper presents a novel approach to equipping LLMs with product knowledge by training them to respond contextually to synthetic search queries that include product IDs. We delve into an extensive analysis of this method, evaluating its effectiveness, outlining its benefits, and highlighting its constraints. The paper also discusses the potential improvements and future directions for this approach, providing a comprehensive understanding of the role of LLMs in product recommendations.

Learn by Selling: Equipping Large Language Models with Product Knowledge for Context-Driven Recommendations

TL;DR

The paper tackles enabling LLMs to deliver context-driven product recommendations by grounding them in a full product inventory. It introduces a data-generation pipeline that creates synthetic search queries embedding product IDs and generates corresponding sales responses, enabling supervised fine-tuning of an LLM. Using the Mistral7Bv0.2 model with product IDs as tokens, the approach improves ranking and category accuracy, but reveals significant challenges in factual accuracy and hallucination, especially for product details like series name and price. The work highlights practical implications for scalable, personalized recommendations and outlines directions to enhance adaptability to new products and reduce misinformation through targeted training and retrieval-based strategies.

Abstract

The rapid evolution of large language models (LLMs) has opened up new possibilities for applications such as context-driven product recommendations. However, the effectiveness of these models in this context is heavily reliant on their comprehensive understanding of the product inventory. This paper presents a novel approach to equipping LLMs with product knowledge by training them to respond contextually to synthetic search queries that include product IDs. We delve into an extensive analysis of this method, evaluating its effectiveness, outlining its benefits, and highlighting its constraints. The paper also discusses the potential improvements and future directions for this approach, providing a comprehensive understanding of the role of LLMs in product recommendations.
Paper Structure (16 sections, 4 figures, 2 tables)

This paper contains 16 sections, 4 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Large Language Models
  4. LLM-based Recommendation
  5. Dataset
  6. Generating Search Queries
  7. Generating the Sales Response
  8. Final Dataset
  9. Methodology
  10. Training
  11. Evaluation
  12. Quantitative
  13. Qualitative:
  14. Results
  15. Does a LLM learn the product information?
  16. ...and 1 more sections

Figures (4)

  • Figure 1: Example of the pipeline from the Product Information to the Sales Response. The LLM fine-tuning is done on the search query and the sales response pair. (Note: The IKEA logo and the IKEA wordmark are registered trademarks of Inter IKEA Systems B.V.)
  • Figure 2: Guided Prompt for generating Sales Response using GPT-4
  • Figure 3: Tailored Sofa Recommendations Based on User Needs: Search Results for Family-Oriented Users (Left) and Budget-Conscious Students (Right).
  • Figure 4: Results of Qualitative Analysis on Mistral(with Product ID tokens)