Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Real-time Bidding Strategy in Display Advertising: An Empirical Analysis

Mengjuan Liu, Zhengning Hu, Zhi Lai, Daiwei Zheng, Xuyun Nie

TL;DR

This work empirically analyzes real-time bidding strategies for display advertising, focusing on reinforcement learning approaches. It formalizes the bidding problem as an episodic MDP and compares model-based (RLB) and model-free (DRLB, FAB) strategies on the iPinYou dataset, highlighting how state, action, and reward choices shape performance. Results show model-free, continuous-action methods (FAB) outperform static baselines and other RL variants, with reward design proving crucial. The authors distill practical guidelines for RL-based RTB systems and provide open-source code to support replication and further research.

Abstract

Bidding strategies that help advertisers determine bidding prices are receiving increasing attention as more and more ad impressions are sold through real-time bidding systems. This paper first describes the problem and challenges of optimizing bidding strategies for individual advertisers in real-time bidding display advertising. Then, several representative bidding strategies are introduced, especially the research advances and challenges of reinforcement learning-based bidding strategies. Further, we quantitatively evaluate the performance of several representative bidding strategies on the iPinYou dataset. Specifically, we examine the effects of state, action, and reward function on the performance of reinforcement learning-based bidding strategies. Finally, we summarize the general steps for optimizing bidding strategies using reinforcement learning algorithms and present our suggestions.

Real-time Bidding Strategy in Display Advertising: An Empirical Analysis

TL;DR

This work empirically analyzes real-time bidding strategies for display advertising, focusing on reinforcement learning approaches. It formalizes the bidding problem as an episodic MDP and compares model-based (RLB) and model-free (DRLB, FAB) strategies on the iPinYou dataset, highlighting how state, action, and reward choices shape performance. Results show model-free, continuous-action methods (FAB) outperform static baselines and other RL variants, with reward design proving crucial. The authors distill practical guidelines for RL-based RTB systems and provide open-source code to support replication and further research.

Abstract

Bidding strategies that help advertisers determine bidding prices are receiving increasing attention as more and more ad impressions are sold through real-time bidding systems. This paper first describes the problem and challenges of optimizing bidding strategies for individual advertisers in real-time bidding display advertising. Then, several representative bidding strategies are introduced, especially the research advances and challenges of reinforcement learning-based bidding strategies. Further, we quantitatively evaluate the performance of several representative bidding strategies on the iPinYou dataset. Specifically, we examine the effects of state, action, and reward function on the performance of reinforcement learning-based bidding strategies. Finally, we summarize the general steps for optimizing bidding strategies using reinforcement learning algorithms and present our suggestions.
Paper Structure (33 sections, 17 equations, 7 figures, 13 tables)

This paper contains 33 sections, 17 equations, 7 figures, 13 tables.

Table of Contents

  1. Introduction
  2. Problem Definition
  3. Real-time Bidding Strategy
  4. RL-based Bidding strategy
  5. Representative Bidding Strategies
  6. Static Bidding strategies
  7. RL-based Bidding strategies
  8. RLB
  9. DRLB
  10. FAB
  11. Unified Form of Bidding Function
  12. Discussions
  13. MDP modeling
  14. State: more or less statistics?
  15. Action: discrete or continuous?
  16. ...and 18 more sections

Figures (7)

  • Figure 1: An illustration of the bidding process in RTB display advertising
  • Figure 2: An illustration of the RL bidding agent interacting with the RTB environment.
  • Figure 3: The average market price per day for training and testing sets.
  • Figure 4: The average market price for each time slot of June 6 and June 13 in dataset 3358.
  • Figure 5: The number of ad impression per day for training and testing sets.
  • ...and 2 more figures