Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Hint-Augmented Re-ranking: Efficient Product Search using LLM-Based Query Decomposition

Yilun Zhu, Nikhita Vedula, Shervin Malmasi

TL;DR

This work tackles interpreting superlatives in e-commerce search by proposing a hint-augmented ranking framework that decomposes superlatives into domain-specific attribute-value hints to guide retrieval and lightweight reranking. It builds a large, specialized dataset of 21k queries and 470k products, with 1.07 million query–item pairs and three-way relevance labels, and demonstrates how offline LLM-generated hints (reasoning, brands, features, and alternative queries) can be transferred to small models to achieve production-scale latency. Across sparse and dense retrieval, plus listwise and pointwise reranking, hint augmentation yields substantial gains in ranking quality, with final 3B-sized, hint-enhanced pointwise models surpassing larger baselines (P@1 up to 64.39) while reducing compute by orders of magnitude. The approach maintains robustness across human and judge-based evaluations and offers a practical pathway to deploying reasoning-enabled search in high-traffic e-commerce settings. This work highlights how explicit, structured interpretations of superlative user queries can bridge natural language preferences and operational search pipelines, enabling efficient yet effective product retrieval and ranking.

Abstract

Search queries with superlatives (e.g., best, most popular) require comparing candidates across multiple dimensions, demanding linguistic understanding and domain knowledge. We show that LLMs can uncover latent intent behind these expressions in e-commerce queries through a framework that extracts structured interpretations or hints. Our approach decomposes queries into attribute-value hints generated concurrently with retrieval, enabling efficient integration into the ranking pipeline. Our method improves search performanc eby 10.9 points in MAP and ranking by 5.9 points in MRR over baselines. Since direct LLM-based reranking faces prohibitive latency, we develop an efficient approach transferring superlative interpretations to lightweight models. Our findings provide insights into how superlative semantics can be represented and transferred between models, advancing linguistic interpretation in retrieval systems while addressing practical deployment constraints.

Hint-Augmented Re-ranking: Efficient Product Search using LLM-Based Query Decomposition

TL;DR

This work tackles interpreting superlatives in e-commerce search by proposing a hint-augmented ranking framework that decomposes superlatives into domain-specific attribute-value hints to guide retrieval and lightweight reranking. It builds a large, specialized dataset of 21k queries and 470k products, with 1.07 million query–item pairs and three-way relevance labels, and demonstrates how offline LLM-generated hints (reasoning, brands, features, and alternative queries) can be transferred to small models to achieve production-scale latency. Across sparse and dense retrieval, plus listwise and pointwise reranking, hint augmentation yields substantial gains in ranking quality, with final 3B-sized, hint-enhanced pointwise models surpassing larger baselines (P@1 up to 64.39) while reducing compute by orders of magnitude. The approach maintains robustness across human and judge-based evaluations and offers a practical pathway to deploying reasoning-enabled search in high-traffic e-commerce settings. This work highlights how explicit, structured interpretations of superlative user queries can bridge natural language preferences and operational search pipelines, enabling efficient yet effective product retrieval and ranking.

Abstract

Search queries with superlatives (e.g., best, most popular) require comparing candidates across multiple dimensions, demanding linguistic understanding and domain knowledge. We show that LLMs can uncover latent intent behind these expressions in e-commerce queries through a framework that extracts structured interpretations or hints. Our approach decomposes queries into attribute-value hints generated concurrently with retrieval, enabling efficient integration into the ranking pipeline. Our method improves search performanc eby 10.9 points in MAP and ranking by 5.9 points in MRR over baselines. Since direct LLM-based reranking faces prohibitive latency, we develop an efficient approach transferring superlative interpretations to lightweight models. Our findings provide insights into how superlative semantics can be represented and transferred between models, advancing linguistic interpretation in retrieval systems while addressing practical deployment constraints.

Paper Structure

This paper contains 38 sections, 2 figures, 15 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Superlatives
  4. LLM Ranking
  5. Datasets for Information Retrieval
  6. Dataset Construction
  7. Superlative Queries
  8. Relevance Labels
  9. Decomposing Superlatives into Hints
  10. Experiments & Evaluation
  11. Retriever Models
  12. Sparse Retrievers
  13. Dense Retriever
  14. Re-ranking Models
  15. Listwise
  16. ...and 23 more sections

Figures (2)

  • Figure 1: The framework of our proposed hint-augmented ranking approach.
  • Figure 2: Relationship between computational cost and MRR performance.