GRIT: Graph-based Recall Improvement for Task-oriented E-commerce Queries

TL;DR

GRIT targets first-stage recall in task-oriented e-commerce queries by constructing a product-product similarity graph from user interaction data and applying a seed-based neighbor expansion during retrieval. To evaluate, the authors create the Task-oriented Query set for E-commerce (TQE) benchmark using LLM-generated task-oriented variants on ESCI data. Empirical results show statistically significant recall gains across lexical, dense, and learned-sparse baselines, with robustness to parameter changes and no extra latency. The work provides a practical benchmark and a scalable graph-based recall mechanism that can improve recall in production e-commerce search pipelines.

Abstract

Many e-commerce search pipelines have four stages, namely: retrieval, filtering, ranking, and personalized-reranking. The retrieval stage must be efficient and yield high recall because relevant products missed in the first stage cannot be considered in later stages. This is challenging for task-oriented queries (queries with actionable intent) where user requirements are contextually intensive and difficult to understand. To foster research in the domain of e-commerce, we created a novel benchmark for Task-oriented Queries (TQE) by using LLM, which operates over the existing ESCI product search dataset. Furthermore, we propose a novel method 'Graph-based Recall Improvement for Task-oriented queries' (GRIT) to address the most crucial first-stage recall improvement needs. GRIT leads to robust and statistically significant improvements over state-of-the-art lexical, dense, and learned-sparse baselines. Our system supports both traditional and task-oriented e-commerce queries, yielding up to 6.3% recall improvement. In the indexing stage, GRIT first builds a product-product similarity graph using user clicks or manual annotation data. During retrieval, it locates neighbors with higher contextual and action relevance and prioritizes them over the less relevant candidates from the initial retrieval. This leads to a more comprehensive and relevant first-stage result set that improves overall system recall. Overall, GRIT leverages the locality relationships and contextual insights provided by the graph using neighboring nodes to enrich the first-stage retrieval results. We show that the method is not only robust across all introduced parameters, but also works effectively on top of a variety of first-stage retrieval methods.

Figures (5)

• Figure 1: Comparison of our approach (GRIT) with baselines in terms of Recall at $n$ results. Solid line: GRIT, Dotted line: baseline for respective baseline. The plots are on traditional ESCI queries and TQE benchmark (small) respectively. It is clearly evident that GRIT significantly improves Recall of initial retrieval across methods, $n$ values and query types.
• Figure 2: TQE: Task-oriented query generation
• Figure 3: GRIT System Architecture
• Figure 4: Recall at first 1000 results for GRIT$t$ parameter values from $t=0.000$ to $t=0.040$ at the interval of $0.005$. Baseline result is at $t=0$ denoted by dot. On ESCI small and large query sets respectively. Robustness of GRIT is clearly evident across seed size parameter ($t$) values on traditional ESCI product search queries. Further, we also note that BM25 performs the best followed by TAS-B.
• Figure 5: Recall at first 1000 results for GRIT across $t$ parameter values from $t=0.000$ to $t=0.040$ at the interval of $0.005$. Baseline result is at $t=0$ denoted by dot. On TQE benchmark - small and large query sets respectively. Robustness of GRIT is clearly evident across seed size parameter ($t$) values on the TQE benchmark. Further, we also note that for the small query set BM25 and TAS-B show comparable performance. But, in case of large query set TAS-B outperforms BM25.