LLMs as Sparse Retrievers:A Framework for First-Stage Product Search

TL;DR

The paper tackles scalable first-stage product search by introducing PROSPER, an LLM-based sparse retrieval framework designed to balance semantic understanding with interpretability and indexing efficiency. It couples a literal residual network (LRN) with a lexical focusing window (LFW) to mitigate lexical expansion hallucinations and to stabilize training in ultra-high-dimensional vocabularies, respectively. Through extensive offline evaluation on Multi-CPR E-commerce and Taobao-Internal, PROSPER outperforms strong sparse baselines and rivals dense retrievers in recall, while online deployment on Taobao demonstrates modest but meaningful revenue uplift. The work provides a practical pathway to deploy interpretable, scalable sparse retrieval with LLMs in large-scale e-commerce pipelines, highlighting a viable alternative to dense representations for first-stage retrieval.

Abstract

Product search is a crucial component of modern e-commerce platforms, with billions of user queries every day. In product search systems, first-stage retrieval should achieve high recall while ensuring efficient online deployment. Sparse retrieval is particularly attractive in this context due to its interpretability and storage efficiency. However, sparse retrieval methods suffer from severe vocabulary mismatch issues, leading to suboptimal performance in product search scenarios. With their potential for semantic analysis, large language models (LLMs) offer a promising avenue for mitigating vocabulary mismatch issues and thereby improving retrieval quality. Directly applying LLMs to sparse retrieval in product search exposes two key challenges:(1)Queries and product titles are typically short and highly susceptible to LLM-induced hallucinations, such as generating irrelevant expansion terms or underweighting critical literal terms like brand names and model numbers;(2)The large vocabulary space of LLMs leads to difficulty in initializing training effectively, making it challenging to learn meaningful sparse representations in such ultra-high-dimensional spaces.To address these challenges, we propose PROSPER, a framework for PROduct search leveraging LLMs as SParsE Retrievers. PROSPER incorporates: (1)A literal residual network that alleviates hallucination in lexical expansion by reinforcing underweighted literal terms through a residual compensation mechanism; and (2)A lexical focusing window that facilitates effective training initialization via a coarse-to-fine sparsification strategy.Extensive offline and online experiments show that PROSPER significantly outperforms sparse baselines and achieves recall performance comparable to advanced dense retrievers, while also achieving revenue increments online.

Figures (7)

• Figure 1: Overview of the proposed PROSPER framework. (a) The model architecture, detailing the query/item encoding process and the core LRN module that enhances literal term weights. (b) The model training, illustrating the LFW, the asymmetric similarity computation, and the final loss function combining ranking loss with FLOPS regularization.
• Figure 2: Impact of the LRN on retrieval performance.
• Figure 3: Analysis of the impact of literal terms and expansion terms on retrieval performance.
• Figure 4: Effectiveness (MRR@10) vs. Efficiency (# FLOPS) trade-off on Multi-CPR E-commerce.
• Figure 5: Overview of the architecture of the Taobao search engine with the proposed sparse retrieval model PROSPER.