AuctionNet: A Novel Benchmark for Decision-Making in Large-Scale Games

TL;DR

AuctionNet tackles the challenge of evaluating bid decision-making in large-scale ad auctions by providing a data-driven benchmark built from a real-world platform. It introduces an ad auction environment with a latent-diffusion–based ad-opportunity generator, a multi-agent bidding module, and a GSP-based auction with optional multi-slot support, along with a large pre-generated dataset (over 10 million opportunities and 500+ million records) and baseline evaluations including PID, Online LP, IQL, BC, and DT. The framework is formalized via a POSG, enabling principled analysis of budgets, values, and strategic interactions, and is demonstrated through validation against real data, rich analytics, and a NeurIPS competition that leveraged the benchmark. AuctionNet aims to accelerate progress in bid decision-making for large-scale games and broader decision-making research in RL, generative modeling, and operations research, by offering a realistic, scalable platform for offline training and evaluation. The work highlights both the practical impact on online advertising and the methodological benefits of data-driven simulation for complex, multi-agent decision problems, while noting limitations in generative-data biases and potential fidelity gaps.

Abstract

Decision-making in large-scale games is an essential research area in artificial intelligence (AI) with significant real-world impact. However, the limited access to realistic large-scale game environments has hindered research progress in this area. In this paper, we present AuctionNet, a benchmark for bid decision-making in large-scale ad auctions derived from a real-world online advertising platform. AuctionNet is composed of three parts: an ad auction environment, a pre-generated dataset based on the environment, and performance evaluations of several baseline bid decision-making algorithms. More specifically, the environment effectively replicates the integrity and complexity of real-world ad auctions through the interaction of several modules: the ad opportunity generation module employs deep generative networks to bridge the gap between simulated and real-world data while mitigating the risk of sensitive data exposure; the bidding module implements diverse auto-bidding agents trained with different decision-making algorithms; and the auction module is anchored in the classic Generalized Second Price (GSP) auction but also allows for customization of auction mechanisms as needed. To facilitate research and provide insights into the environment, we have also pre-generated a substantial dataset based on the environment. The dataset contains 10 million ad opportunities, 48 diverse auto-bidding agents, and over 500 million auction records. Performance evaluations of baseline algorithms such as linear programming, reinforcement learning, and generative models for bid decision-making are also presented as a part of AuctionNet. We believe that AuctionNet is applicable not only to research on bid decision-making in ad auctions but also to the general area of decision-making in large-scale games.

Figures (10)

• Figure 1: Overview of typical large-scale online advertising platform. Numbers 1 through 5 illustrate how an auto-bidding agent helps advertiser $i$ optimize performance. For each advertiser's unique objective (I), auto-bidding agent make bid decision-making (II) for continuously arriving ad opportunities, and compete against each other in the ad auction (III). Then, each agent may win some impressions (IV), which may be exposed to users and potentially result in conversions. Finally, the agents' performance (V) will be reported to advertisers.
• Figure 2: Overview of the pipeline of the ad opportunity generation network. The generation process consists of two stages. In the first stage, ad opportunity features are generated through a latent diffusion model. In the second stage, the value prediction for the generated ad opportunity features is performed, incorporating both the time feature and the advertiser feature. Moreover, the volume of ad opportunities fluctuates over time, mirroring that of real-world online advertising platforms.
• Figure 3: The 3D PCA results of 100K generated data and 100K real-world data.
• Figure 4: The distribution of identity information including the Taobao VIP level, the preferred phone price, the buyer level, and the gender in 100K generated data and 100K real-world data.
• Figure 5: The distribution of consumption behavior information including the number of collected items, the number of visited items, the number of collected sellers, and the consumption amounts in 100K generated data and 100K real-world data.