Large Language Model based Long-tail Query Rewriting in Taobao Search

TL;DR

This work tackles the semantic gap in e-commerce search for long-tail queries by introducing BEQUE, a three-stage framework that fine-tunes a large language model through multi-instruction supervised training, offline feedback from Taobao's retrieval system, and objective alignment with online goals using a Bradley-Terry–based ranking objective. The approach generates multiple candidate rewrites for each query, evaluates them offline with relevance, increment, and hitrate signals, and aligns generation with online objectives to improve retrieval performance. Offline experiments demonstrate improved rewrite quality and alignment, while online A/B tests on Taobao show measurable gains in GMV, transaction count, and unique visitors—especially for long-tail and few-recall queries. BEQUE is deployed in Taobao since 2023, highlighting a practical, scalable path for production-grade query rewriting in large-scale e-commerce search systems.

Abstract

In the realm of e-commerce search, the significance of semantic matching cannot be overstated, as it directly impacts both user experience and company revenue. Along this line, query rewriting, serving as an important technique to bridge the semantic gaps inherent in the semantic matching process, has attached wide attention from the industry and academia. However, existing query rewriting methods often struggle to effectively optimize long-tail queries and alleviate the phenomenon of "few-recall" caused by semantic gap. In this paper, we present BEQUE, a comprehensive framework that Bridges the sEmantic gap for long-tail QUEries. In detail, BEQUE comprises three stages: multi-instruction supervised fine tuning (SFT), offline feedback, and objective alignment. We first construct a rewriting dataset based on rejection sampling and auxiliary tasks mixing to fine-tune our large language model (LLM) in a supervised fashion. Subsequently, with the well-trained LLM, we employ beam search to generate multiple candidate rewrites, and feed them into Taobao offline system to obtain the partial order. Leveraging the partial order of rewrites, we introduce a contrastive learning method to highlight the distinctions between rewrites, and align the model with the Taobao online objectives. Offline experiments prove the effectiveness of our method in bridging semantic gap. Online A/B tests reveal that our method can significantly boost gross merchandise volume (GMV), number of transaction (#Trans) and unique visitor (UV) for long-tail queries. BEQUE has been deployed on Taobao, one of most popular online shopping platforms in China, since October 2023.

Figures (2)

• Figure 1: Framework of Taobao search engine.
• Figure 2: Framework of BEQUE.