Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Optimal No-regret Learning in Repeated First-price Auctions

Yanjun Han, Zhengyuan Zhou, Tsachy Weissman

TL;DR

The first learning algorithm that achieves a near-optimal regret bound is developed, by exploiting two structural properties of first-price auctions, that is, the specific feedback structure and payoff function.

Abstract

We study online learning in repeated first-price auctions where a bidder, only observing the winning bid at the end of each auction, learns to adaptively bid in order to maximize her cumulative payoff. To achieve this goal, the bidder faces censored feedback: if she wins the bid, then she is not able to observe the highest bid of the other bidders, which we assume is \textit{iid} drawn from an unknown distribution. In this paper, we develop the first learning algorithm that achieves a near-optimal $\widetilde{O}(\sqrt{T})$ regret bound, by exploiting two structural properties of first-price auctions, i.e. the specific feedback structure and payoff function. We first formulate the feedback structure in first-price auctions as partially ordered contextual bandits, a combination of the graph feedback across actions (bids), the cross learning across contexts (private values), and a partial order over the contexts. We establish both strengths and weaknesses of this framework, by showing a curious separation that a regret nearly independent of the action/context sizes is possible under stochastic contexts, but is impossible under adversarial contexts. In particular, this framework leads to an $O(\sqrt{T}\log^{2.5}T)$ regret for first-price auctions when the bidder's private values are \emph{iid}. Despite the limitation of the above framework, we further exploit the special payoff function of first-price auctions to develop a sample-efficient algorithm even in the presence of adversarially generated private values. We establish an $O(\sqrt{T}\log^3 T)$ regret bound for this algorithm, hence providing a complete characterization of optimal learning guarantees for first-price auctions.

Optimal No-regret Learning in Repeated First-price Auctions

TL;DR

The first learning algorithm that achieves a near-optimal regret bound is developed, by exploiting two structural properties of first-price auctions, that is, the specific feedback structure and payoff function.

Abstract

We study online learning in repeated first-price auctions where a bidder, only observing the winning bid at the end of each auction, learns to adaptively bid in order to maximize her cumulative payoff. To achieve this goal, the bidder faces censored feedback: if she wins the bid, then she is not able to observe the highest bid of the other bidders, which we assume is \textit{iid} drawn from an unknown distribution. In this paper, we develop the first learning algorithm that achieves a near-optimal regret bound, by exploiting two structural properties of first-price auctions, i.e. the specific feedback structure and payoff function. We first formulate the feedback structure in first-price auctions as partially ordered contextual bandits, a combination of the graph feedback across actions (bids), the cross learning across contexts (private values), and a partial order over the contexts. We establish both strengths and weaknesses of this framework, by showing a curious separation that a regret nearly independent of the action/context sizes is possible under stochastic contexts, but is impossible under adversarial contexts. In particular, this framework leads to an regret for first-price auctions when the bidder's private values are \emph{iid}. Despite the limitation of the above framework, we further exploit the special payoff function of first-price auctions to develop a sample-efficient algorithm even in the presence of adversarially generated private values. We establish an regret bound for this algorithm, hence providing a complete characterization of optimal learning guarantees for first-price auctions.

Paper Structure

This paper contains 37 sections, 27 theorems, 101 equations, 1 figure, 4 algorithms.

Table of Contents

  1. Introduction
  2. Key Challenge
  3. Our Algorithms and Results
  4. Stochastic private values
  5. Adversarial private values
  6. Related Work
  7. Problem Formulation and Main Results
  8. Notation
  9. Problem Setup
  10. Main Results
  11. Organization
  12. Learning in First-price Auctions with Stochastic Private Values
  13. An Algorithm for Stochastic Multi-armed Bandits with Graph Feedback
  14. Stochastic Contextual Bandits with Graph Feedback and Cross Learning
  15. Partially Ordered Contextual Bandits
  16. ...and 22 more sections

Key Result

Theorem 1

Let $v_1,\cdots,v_T$ be iid drawn from any unknown distribution $F$. Then there exists a bidding policy $\pi$ (Algorithm algo.partial_order applied to first-price auctions, see Corollary cor.stochastic) satisfying where the expectation is taken jointly over the randomness of $v$ and the policy $\pi$, and $C>0$ is an absolute constant independent of the time horizon $T$ and the CDFs $(F,G)$.

Figures (1)

  • Figure 1: Panel (a) provides a simplified schematic diagram first-price display ads auction. Panel (b) shows the spending change for bidders' on AppNexus news7, which switched from second-price to first-price auctions in 2018.

Theorems & Definitions (43)

  • Theorem 1: Bidding with stochastic private values
  • Theorem 2: Bidding with arbitrary private values
  • Definition 1: Stochastic Multi-armed Bandits with Graph Feedback
  • Theorem 3
  • Lemma 1
  • proof : Proof of Theorem \ref{['thm.MAB_feedback']}
  • Definition 2: Stochastic Contextual Bandits with Graph Feedback and Cross Learning
  • Theorem 4
  • proof : Proof of Theorem \ref{['thm.contextual']}
  • Remark 1
  • ...and 33 more