Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Bid2X: Revealing Dynamics of Bidding Environment in Online Advertising from A Foundation Model Lens

Jiahao Ji, Tianyu Wang, Yeshu Li, Yushen Huo, Zhilin Zhang, Chuan Yu, Jian Xu, Bo Zheng

TL;DR

Bid2X tackles the challenge of generalizing bidding environment modeling across diverse online advertising scenarios by learning a foundation model from broad bidding data. It unifies heterogeneous bidding data into series embeddings, processes them with a Bid2X Transformer that uses variable-attention and temporal-attention streams, and applies a variable-aware fusion to produce bid-outcome predictions. To match bid data's zero-inflated distribution, it introduces a zero-inflated projection that jointly models the probability of non-zero outcomes and their magnitudes, with an auxiliary cumulative-prediction task and a theoretical guarantee of convergence. Empirically, Bid2X achieves superior generalization across eight real-world datasets, and online Taobao deployments report GMV and ROI improvements, illustrating the practicality of a bidding foundation model in computational advertising.

Abstract

Auto-bidding is crucial in facilitating online advertising by automatically providing bids for advertisers. While previous work has made great efforts to model bidding environments for better ad performance, it has limitations in generalizability across environments since these models are typically tailored for specific bidding scenarios. To this end, we approach the scenario-independent principles through a unified function that estimates the achieved effect under specific bids, such as budget consumption, gross merchandise volume (GMV), page views, etc. Then, we propose a bidding foundation model Bid2X to learn this fundamental function from data in various scenarios. Our Bid2X is built over uniform series embeddings that encode heterogeneous data through tailored embedding methods. To capture complex inter-variable and dynamic temporal dependencies in bidding data, we propose two attention mechanisms separately treating embeddings of different variables and embeddings at different times as attention tokens for representation learning. On top of the learned variable and temporal representations, a variable-aware fusion module is used to perform adaptive bidding outcome prediction. To model the unique bidding data distribution, we devise a zero-inflated projection module to incorporate the estimated non-zero probability into its value prediction, which makes up a joint optimization objective containing classification and regression. The objective is proven to converge to the zero-inflated distribution. Our model has been deployed on the ad platform in Taobao, one of the world's largest e-commerce platforms. Offline evaluation on eight datasets exhibits Bid2X's superiority compared to various baselines and its generality across different scenarios. Bid2X increased GMV by 4.65% and ROI by 2.44% in online A/B tests, paving the way for bidding foundation model in computational advertising.

Bid2X: Revealing Dynamics of Bidding Environment in Online Advertising from A Foundation Model Lens

TL;DR

Bid2X tackles the challenge of generalizing bidding environment modeling across diverse online advertising scenarios by learning a foundation model from broad bidding data. It unifies heterogeneous bidding data into series embeddings, processes them with a Bid2X Transformer that uses variable-attention and temporal-attention streams, and applies a variable-aware fusion to produce bid-outcome predictions. To match bid data's zero-inflated distribution, it introduces a zero-inflated projection that jointly models the probability of non-zero outcomes and their magnitudes, with an auxiliary cumulative-prediction task and a theoretical guarantee of convergence. Empirically, Bid2X achieves superior generalization across eight real-world datasets, and online Taobao deployments report GMV and ROI improvements, illustrating the practicality of a bidding foundation model in computational advertising.

Abstract

Auto-bidding is crucial in facilitating online advertising by automatically providing bids for advertisers. While previous work has made great efforts to model bidding environments for better ad performance, it has limitations in generalizability across environments since these models are typically tailored for specific bidding scenarios. To this end, we approach the scenario-independent principles through a unified function that estimates the achieved effect under specific bids, such as budget consumption, gross merchandise volume (GMV), page views, etc. Then, we propose a bidding foundation model Bid2X to learn this fundamental function from data in various scenarios. Our Bid2X is built over uniform series embeddings that encode heterogeneous data through tailored embedding methods. To capture complex inter-variable and dynamic temporal dependencies in bidding data, we propose two attention mechanisms separately treating embeddings of different variables and embeddings at different times as attention tokens for representation learning. On top of the learned variable and temporal representations, a variable-aware fusion module is used to perform adaptive bidding outcome prediction. To model the unique bidding data distribution, we devise a zero-inflated projection module to incorporate the estimated non-zero probability into its value prediction, which makes up a joint optimization objective containing classification and regression. The objective is proven to converge to the zero-inflated distribution. Our model has been deployed on the ad platform in Taobao, one of the world's largest e-commerce platforms. Offline evaluation on eight datasets exhibits Bid2X's superiority compared to various baselines and its generality across different scenarios. Bid2X increased GMV by 4.65% and ROI by 2.44% in online A/B tests, paving the way for bidding foundation model in computational advertising.

Paper Structure

This paper contains 33 sections, 2 theorems, 17 equations, 7 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Preliminaries
  4. Method
  5. Unified Data Embedding
  6. Bidding Transformer
  7. Variable Attention-based Encoder
  8. Temporal Attention-based Decoder
  9. Variable-aware Fusion
  10. Zero-inflated Projection & Auxiliary Task
  11. Zero-inflated Projection
  12. Cumulative Prediction
  13. Model Training
  14. Theoretical Analysis
  15. Experiments
  16. ...and 18 more sections

Key Result

Theorem 4

Let $\mathcal{F}$ be a set of all measurable functions and $p_{\theta}, g_{\phi} \in \mathcal{F}$. Assume that $p$ and $g$ are conditionally independent, then, with respect to the zero-inflated model defined in Eq. eq:zinf, we have $p_{\theta^*}(\bm{x}) = p(\bm{x})$ and $p_{\theta^*}(\bm{x})g_{\phi^

Figures (7)

  • Figure 1: Existing models are designed and trained for specific bidding scenarios, so they cannot adapt to various scenarios. In contrast, we propose to train a one-for-all foundation model on broad bidding data, enabling it to handle a wide range of bidding scenarios.
  • Figure 2: The overall architecture of our Bid2X model.
  • Figure 3: Few-shot performance of Bid2X and baselines on the crowd-type bidding scenario with 1% and 5% training data. Zero-shot results of our Bid2X are given by the dashed line.
  • Figure 4: Bidding environment modeling performance improves smoothly as we increase the (a) dataset size and (b) model size. (c) Training loss curve of varying-size models.
  • Figure 5: Our model's adherence to physics laws.
  • ...and 2 more figures

Theorems & Definitions (5)

  • Definition 1: Ad Campaign
  • Definition 2: Bidding Trajectory
  • Definition 3: Bidding Environment Modeling Problem
  • Theorem 4: Zero-inflated Proper Loss
  • Theorem 3: Zero-inflated Proper Loss