Middleman Bias in Advertising: Aligning Relevance of Keyphrase Recommendations with Search
Soumik Dey, Wei Zhang, Hansi Wu, Bingfeng Dong, Binbin Li
TL;DR
This work addresses middleman bias in advertiser keyphrase relevance by reframing the problem as an interaction between Advertising and the Search middleman. It compares bi-encoder and cross-encoder architectures, trained on a curated dataset of Search relevance judgments rather than biased click data, to align Advertising recommendations with Search relevance. Offline results show that cross-encoders, particularly small models like bert-mini and bert-tiny, outperform bi-encoders in relevance metrics, and online deployment of a lightweight cross-encoder yields substantial gains in impressions, clicks, and conversion rate while reducing false positives and negatives. The study demonstrates a scalable, low-latency approach to align relevance signals across cascade ranking systems in e-commerce advertising and highlights practical implications for improving auction outcomes and user satisfaction.
Abstract
E-commerce sellers are recommended keyphrases based on their inventory on which they advertise to increase buyer engagement (clicks/sales). Keyphrases must be pertinent to items; otherwise, it can result in seller dissatisfaction and poor targeting -- towards that end relevance filters are employed. In this work, we describe the shortcomings of training relevance filter models on biased click/sales signals. We re-conceptualize advertiser keyphrase relevance as interaction between two dynamical systems -- Advertising which produces the keyphrases and Search which acts as a middleman to reach buyers. We discuss the bias of search relevance systems (middleman bias) and the need to align advertiser keyphrases with search relevance signals. We also compare the performance of cross encoders and bi-encoders in modeling this alignment and the scalability of such a solution for sellers at eBay.