Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LLM-based Semantic Search for Conversational Queries in E-commerce

Emad Siddiqui, Venkatesh Terikuti, Xuan Lu

TL;DR

This work tackles the challenge of conversational search in e-commerce by integrating LLM-generated synthetic data to fine-tune a domain-specific embedding that captures semantic product relationships, with a separate generative model that converts natural-language queries into structured constraints. The framework combines similarity-based retrieval with constraint-based filtering, implemented via FAISS indexing and a Flan-T5-small-based filter extractor, to improve precision and recall on a real-world catalog. Evaluations on a held-out test set derived from the Amazon ESCI data show meaningful gains over diverse baselines, and the authors release a large synthetic-query dataset and a constrained evaluation benchmark to support future research. The approach demonstrates practical impact for scalable, constraint-aware semantic search in e-commerce, and provides a flexible architecture that can extend across categories and catalogs with adjustable threshold mappings for qualitative constraints.

Abstract

Conversational user queries are increasingly challenging traditional e-commerce platforms, whose search systems are typically optimized for keyword-based queries. We present an LLM-based semantic search framework that effectively captures user intent from conversational queries by combining domain-specific embeddings with structured filters. To address the challenge of limited labeled data, we generate synthetic data using LLMs to guide the fine-tuning of two models: an embedding model that positions semantically similar products close together in the representation space, and a generative model for converting natural language queries into structured constraints. By combining similarity-based retrieval with constraint-based filtering, our framework achieves strong precision and recall across various settings compared to baseline approaches on a real-world dataset.

LLM-based Semantic Search for Conversational Queries in E-commerce

TL;DR

This work tackles the challenge of conversational search in e-commerce by integrating LLM-generated synthetic data to fine-tune a domain-specific embedding that captures semantic product relationships, with a separate generative model that converts natural-language queries into structured constraints. The framework combines similarity-based retrieval with constraint-based filtering, implemented via FAISS indexing and a Flan-T5-small-based filter extractor, to improve precision and recall on a real-world catalog. Evaluations on a held-out test set derived from the Amazon ESCI data show meaningful gains over diverse baselines, and the authors release a large synthetic-query dataset and a constrained evaluation benchmark to support future research. The approach demonstrates practical impact for scalable, constraint-aware semantic search in e-commerce, and provides a flexible architecture that can extend across categories and catalogs with adjustable threshold mappings for qualitative constraints.

Abstract

Conversational user queries are increasingly challenging traditional e-commerce platforms, whose search systems are typically optimized for keyword-based queries. We present an LLM-based semantic search framework that effectively captures user intent from conversational queries by combining domain-specific embeddings with structured filters. To address the challenge of limited labeled data, we generate synthetic data using LLMs to guide the fine-tuning of two models: an embedding model that positions semantically similar products close together in the representation space, and a generative model for converting natural language queries into structured constraints. By combining similarity-based retrieval with constraint-based filtering, our framework achieves strong precision and recall across various settings compared to baseline approaches on a real-world dataset.
Paper Structure (23 sections, 2 equations, 2 figures, 5 tables)

This paper contains 23 sections, 2 equations, 2 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Dataset
  4. Methods
  5. Product Embedding Generation
  6. Sentence transformer selection.
  7. Synthetic query generation.
  8. Model fine-tuning.
  9. Indexing embeddings
  10. User Query Understanding
  11. Structured filter generation for synthetic queries
  12. Model fine-tuning
  13. Evaluation
  14. Test Setting
  15. Baselines
  16. ...and 8 more sections

Figures (2)

  • Figure 1: The LLM-based Semantic Search Framework for Conversational Queries. For each input user query, the framework outputs a ranked list of relevant products by combining similarity search with constraint-based filtering. Embedding component: this component fine-tunes a Sentence Transformer using synthetic queries generated by an LLM from the product catalog. The fine-tuned model is then used to compute embeddings for both the products and the user query. Structure component: this component fine-tunes a generative model to extract numerical and categorical information from user queries and convert it into structured filters. Note that the target product catalog used to generate product embeddings may differ from the catalog used to train the embedding component.
  • Figure 2: Prompt to generate synthetic queries for products. The <PRODUCTS> placeholder is replaced with a list of products, each described by its unique identifier (i.e., parent_asin), product title, features, description, and technical specifications.