Learning to Bid in Non-Stationary Repeated First-Price Auctions

TL;DR

The paper addresses bidding in non-stationary online first-price auctions, where opponents’ bids evolve over time. It develops a dynamic benchmark and two regularity measures $V_T$ and $L_T$ to capture non-stationarity, and shows minimax-optimal dynamic regret rates $\tilde{O}(\sqrt{T V_T})$ and $\tilde{O}(L_T)$ using Optimistic Mirror Descent with a novel optimism $\mu_t=\max\{v_t-m_t,0\}$ and adaptive restart schemes. It provides matching lower bounds and a best-of-both-worlds guarantee via a meta-algorithm, with extensive synthetic experiments demonstrating performance gains over baselines such as Hedge and SEW, particularly under budget-constrained opponents. The results advance understanding of dynamic learning in non-stationary ad-auction environments and offer practical, adaptive bidding strategies for revenue-maximization in real-time auctions.

Abstract

First-price auctions have recently gained significant traction in digital advertising markets, exemplified by Google's transition from second-price to first-price auctions. Unlike in second-price auctions, where bidding one's private valuation is a dominant strategy, determining an optimal bidding strategy in first-price auctions is more complex. From a learning perspective, the learner (a specific bidder) can interact with the environment (other bidders, i.e., opponents) sequentially to infer their behaviors. Existing research often assumes specific environmental conditions and benchmarks performance against the best fixed policy (static benchmark). While this approach ensures strong learning guarantees, the static benchmark can deviate significantly from the optimal strategy in environments with even mild non-stationarity. To address such scenarios, a dynamic benchmark--representing the sum of the highest achievable rewards at each time step--offers a more suitable objective. However, achieving no-regret learning with respect to the dynamic benchmark requires additional constraints. By inspecting reward functions in online first-price auctions, we introduce two metrics to quantify the regularity of the sequence of opponents' highest bids, which serve as measures of non-stationarity. We provide a minimax-optimal characterization of the dynamic regret for the class of sequences of opponents' highest bids that satisfy either of these regularity constraints. Our main technical tool is the Optimistic Mirror Descent (OMD) framework with a novel optimism configuration, which is well-suited for achieving minimax-optimal dynamic regret rates in this context. We then use synthetic datasets to validate our theoretical guarantees and demonstrate that our methods outperform existing ones.

Figures (5)

• Figure 1: An illustration of the construction for the lower bound. In odd-numbered batches, $m_t$ jumps from $0$ to $\frac{1}{H}$ at random locations, while in even-numbered batches, $m_t$ jumps from $\frac{1}{H}$ back to $0$. The parameter $H$ is carefully chosen to satisfy the variation budget constraint and ensure that we obtain the desired lower bound.
• Figure 2: Slopes of dynamic regret rates against varying time horizons for different opponents' highest bid sequence patterns and values of $\alpha$.
• Figure 3: Cumulative rewards against budget-pacing bidders in the sufficient budget regime.
• Figure 4: Cumulative rewards against budget-pacing bidders in the insufficient budget regime.
• Figure 5: Cumulative rewards against budget-pacing bidders for different initial budgets.

Theorems & Definitions (21)

• Example 1
• Remark 1
• Proposition 1
• Example 2
• Example 3
• Proposition 2
• Example 4
• Proposition 3
• Proposition 4
• Example 5