Identifying High Consideration E-Commerce Search Queries

TL;DR

This work introduces an accurate and scalable method for EQR, focusing on query ranking to indicate potential engagement levels with query-related shopping knowledge content during product search, and presents experimental results demonstrating its effectiveness.

Abstract

In e-commerce, high consideration search missions typically require careful and elaborate decision making, and involve a substantial research investment from customers. We consider the task of identifying High Consideration (HC) queries. Identifying such queries enables e-commerce sites to better serve user needs using targeted experiences such as curated QA widgets that help users reach purchase decisions. We explore the task by proposing an Engagement-based Query Ranking (EQR) approach, focusing on query ranking to indicate potential engagement levels with query-related shopping knowledge content during product search. Unlike previous studies on predicting trends, EQR prioritizes query-level features related to customer behavior, finance, and catalog information rather than popularity signals. We introduce an accurate and scalable method for EQR and present experimental results demonstrating its effectiveness. Offline experiments show strong ranking performance. Human evaluation shows a precision of 96% for HC queries identified by our model. The model was commercially deployed, and shown to outperform human-selected queries in terms of downstream customer impact, as measured through engagement.

Figures (3)

• Figure 1: An end-to-end demonstration of how the question-answer pairs of a widget is curated.
• Figure 2: Feature importance vales obtained from XGBoost.
• Figure 3: Zero-shot prompt used for generating engagement score and reasons using OpenAI GPT models.