Autobidding Arena: unified evaluation of the classical and RL-based autobidding algorithms

TL;DR

The paper addresses the need for fair, reproducible evaluation of autobidding algorithms in large-scale auctions. It introduces the Autobidding Arena, an open, environment-driven benchmarking framework built on Alibaba-like data to compare classical and RL-based methods using unified, transparent metrics that capture performance, cost, and budget pacing. Across extensive experiments, well-tuned classical autobidding methods often outperform RL baselines, while RL approaches like CBRL can excel on certain objectives, highlighting the importance of careful hyperparameter and reward design. The work provides a practical, domain-agnostic evaluation protocol with an open-source environment that supports broad applicability and reproducible comparisons for practitioners and researchers alike.

Abstract

Advertisement auctions play a crucial role in revenue generation for e-commerce companies. To make the bidding procedure scalable to thousands of auctions, the automatic bidding (autobidding) algorithms are actively developed in the industry. Therefore, the fair and reproducible evaluation of autobidding algorithms is an important problem. We present a standardized and transparent evaluation protocol for comparing classical and reinforcement learning (RL) autobidding algorithms. We consider the most efficient autobidding algorithms from different classes, e.g., ones based on the controllers, RL, optimal formulas, etc., and benchmark them in the bidding environment. We utilize the most recent open-source environment developed in the industry, which accurately emulates the bidding process. Our work demonstrates the most promising use cases for the considered autobidding algorithms, highlights their surprising drawbacks, and evaluates them according to multiple metrics. We select the evaluation metrics that illustrate the performance of the autobidding algorithms, the corresponding costs, and track the budget pacing. Such a choice of metrics makes our results applicable to the broad range of platforms where autobidding is effective. The presented comparison results help practitioners to evaluate the candidate autobidding algorithms from different perspectives and select ones that are efficient according to their companies' targets.

Figures (5)

• Figure 1: The pipeline of interaction between the seller and the environment at each timestamp. The seller sets an available budget, $B$, and an upper bound on the cost-per-click, $CPC$. Then, the environment reports CTR and CVR values, and the autobidding algorithm on the seller side produces bids. Based on the received bids, the environment evaluates the auction result and its outcome, including clicks and conversions.
• Figure 2: The performance of the classical algorithms tunes on the $AWR$ metric. MPID shows the largest $AWR$ and the smallest $RMSE$ while underperform in terms of other metrics.
• Figure 3: The performance of the classical algorithms tuned on the total number of clicks. Linear and OPT show the largest number of clicks and the smallest eCPC, while underperforming in terms of other metrics.
• Figure 4: The performance of the classical algorithms tuned on the total number of conversions. Linear and PID show the largest number of conversions, while underperforming in terms of other metrics.
• Figure 5: The performance of the classical algorithms tuned on the $RMSE$ metric. MPID and PID give the smallest $RMSE$ while underperforming in terms of other metrics.