VALUE: Value-Aware Large Language Model for Query Rewriting via Weighted Trie in Sponsored Search

TL;DR

VALUE addresses the challenge of aligning semantic rewrite quality with commercial value in sponsored search query rewriting. It introduces a Weighted Trie to inject real-time bidword value into decoding and a WDPO-based alignment to emphasize high-value pairs while preserving relevance. Offline experiments show improvements in semantic relevance and eCPM, and online A/B tests report gains in cost efficiency and RPM, culminating in deployment in a live system. The work demonstrates robust, real-time value-aware generation suitable for dynamic keyword markets.

Abstract

Query-to-bidword(i.e., bidding keyword) rewriting is fundamental to sponsored search, transforming noisy user queries into semantically relevant and commercially valuable keywords. Recent advances in large language models (LLMs) improve semantic relevance through generative retrieval frameworks, but they rarely encode the commercial value of keywords. As a result, rewrites are often semantically correct yet economically suboptimal, and a reinforcement learning from human feedback (RLHF) stage is usually added after supervised fine-tuning(SFT) to mitigate this deficiency. However, conventional preference alignment frequently overemphasize the ordering of bidword values and is susceptible to overfitting, which degrades rewrite quality. In addition, bidword value changes rapidly, while existing generative methods do not respond to these fluctuations. To address this shortcoming, we introduce VALUE(Value-Aware Large language model for qUery rewriting via wEighted trie), a framework that integrates value awareness directly into generation and enhances value alignment during training. VALUE employs the Weighted Trie, a novel variant of the classical trie that stores real-time value signals for each token. During decoding, the framework adjusts the LLM's token probabilities with these signals, constraining the search space and steering generation toward high-value rewrites. The alignment stage uses a fine-grained preference learning strategy that emphasizes stable, high-value differences and down-weights noisy or transient fluctuations, thereby improving robustness and reducing overfitting. Offline experiments show that VALUE significantly outperforms baselines in both semantic matching and value-centric metrics. VALUE has been deployed on our advertising system since October 2024 and served the Double Eleven promotions, the biggest shopping carnival in China.

Figures (4)

• Figure 1: System Overview of our proposed VALUE Framework includes three parts: "SFT and Alignment", "Weighted Trie Construction and Weight Momentum Update", and "Collaborative Inference". During the LLM decoding process, we derive a value distribution from a Weighted Trie to constrain the generation space, and subsequently merge this with another distribution to produce the final output distribution.
• Figure 2: Collaborative Inference of VALUE framework. When generating the next token, we obtain two output distribution from the LLM and Weighted Trie. We exclude tokens not present in the value distribution and adjust the probabilities of the remaining tokens. Then, we sample the next token from the modified distribution.
• Figure 3: Trade-off between RPM and PV rele through $\Theta$. The specific values of RPM and PV rele are confidential, so the RPM and PV rele here are equivalently scaled.
• Figure 4: Case Study of VALUE Framework