PRECTR: A Synergistic Framework for Integrating Personalized Search Relevance Matching and CTR Prediction

TL;DR

PRECTR addresses the challenge of decoupled CTR prediction and text relevance models by introducing a unified framework based on conditional probability fusion that jointly models CTR and relevance. It integrates a Relevance Score Level (RSL) module with a Base CTR module, employs two-stage training and semantic consistency regularization to stabilize learning and restrain irrelevant recommendations, and adds a Personalized Relevance Incentive module to tailor rankings to individual user preferences. Through offline experiments on a production-scale Xianyu dataset and online A/B testing, PRECTR demonstrates improvements in AUC, GAUC, and relevance metrics, as well as gains in CTR and GMV, validating its practical impact. The approach highlights the value of tightly coupling relevance signaling and user personalization within a single fusion framework for more effective and user-specific search ranking in industrial systems.

Abstract

The two primary tasks in the search recommendation system are search relevance matching and click-through rate (CTR) prediction -- the former focuses on seeking relevant items for user queries whereas the latter forecasts which item may better match user interest. Prior research typically develops two models to predict the CTR and search relevance separately, then ranking candidate items based on the fusion of the two outputs. However, such a divide-and-conquer paradigm creates the inconsistency between different models. Meanwhile, the search relevance model mainly concentrates on the degree of objective text matching while neglecting personalized differences among different users, leading to restricted model performance. To tackle these issues, we propose a unified Personalized Search RElevance Matching and CTR Prediction Fusion Model(PRECTR). Specifically, based on the conditional probability fusion mechanism, PRECTR integrates the CTR prediction and search relevance matching into one framework to enhance the interaction and consistency of the two modules. However, directly optimizing CTR binary classification loss may bring challenges to the fusion model's convergence and indefinitely promote the exposure of items with high CTR, regardless of their search relevance. Hence, we further introduce two-stage training and semantic consistency regularization to accelerate the model's convergence and restrain the recommendation of irrelevant items. Finally, acknowledging that different users may have varied relevance preferences, we assessed current users' relevance preferences by analyzing past users' preferences for similar queries and tailored incentives for different candidate items accordingly. Extensive experimental results on our production dataset and online A/B testing demonstrate the effectiveness and superiority of our proposed PRECTR method.

Figures (4)

• Figure 1: An illustration of the semantic consistency regularization.
• Figure 2: The overall framework of PRECTR can be broadly divided into three core components: the Base module, the Rsl module, and the Personalized Relevance Incentive module. The Base module and the Rsl module compute $P(click=1|x,rsl)$ and $P(rsl|x)$ respectively and end up with the original score, while the Personalized Relevance Incentive module computes the incentive score $\tau$ based on the user's personalized search relevance preference and finally generate the predicated CTR.
• Figure 3: The effect of the hyperparameter $\alpha$ in semantic consistency regularization.
• Figure 4: The effect of the hyperparameter $\gamma$ in the final optimization objective.