Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Statistical Inference and A/B Testing in Fisher Markets and Paced Auctions

Luofeng Liao, Christian Kroer

Abstract

We initiate the study of statistical inference and A/B testing for two market equilibrium models: linear Fisher market (LFM) equilibrium and first-price pacing equilibrium (FPPE). LFM arises from fair resource allocation systems such as allocation of food to food banks and notification opportunities to different types of notifications. For LFM, we assume that the data observed is captured by the classical finite-dimensional Fisher market equilibrium, and its steady-state behavior is modeled by a continuous limit Fisher market. The second type of equilibrium we study, FPPE, arises from internet advertising where advertisers are constrained by budgets and advertising opportunities are sold via first-price auctions. For platforms that use pacing-based methods to smooth out the spending of advertisers, FPPE provides a hindsight-optimal configuration of the pacing method. We propose a statistical framework for the FPPE model, in which a continuous limit FPPE models the steady-state behavior of the auction platform, and a finite FPPE provides the data to estimate primitives of the limit FPPE. Both LFM and FPPE have an Eisenberg-Gale convex program characterization, the pillar upon which we derive our statistical theory. We start by deriving basic convergence results for the finite market to the limit market. We then derive asymptotic distributions, and construct confidence intervals. Furthermore, we establish the asymptotic local minimax optimality of estimation based on finite markets. We then show that the theory can be used for conducting statistically valid A/B testing on auction platforms. Synthetic and semi-synthetic experiments verify the validity and practicality of our theory.

Statistical Inference and A/B Testing in Fisher Markets and Paced Auctions

Abstract

We initiate the study of statistical inference and A/B testing for two market equilibrium models: linear Fisher market (LFM) equilibrium and first-price pacing equilibrium (FPPE). LFM arises from fair resource allocation systems such as allocation of food to food banks and notification opportunities to different types of notifications. For LFM, we assume that the data observed is captured by the classical finite-dimensional Fisher market equilibrium, and its steady-state behavior is modeled by a continuous limit Fisher market. The second type of equilibrium we study, FPPE, arises from internet advertising where advertisers are constrained by budgets and advertising opportunities are sold via first-price auctions. For platforms that use pacing-based methods to smooth out the spending of advertisers, FPPE provides a hindsight-optimal configuration of the pacing method. We propose a statistical framework for the FPPE model, in which a continuous limit FPPE models the steady-state behavior of the auction platform, and a finite FPPE provides the data to estimate primitives of the limit FPPE. Both LFM and FPPE have an Eisenberg-Gale convex program characterization, the pillar upon which we derive our statistical theory. We start by deriving basic convergence results for the finite market to the limit market. We then derive asymptotic distributions, and construct confidence intervals. Furthermore, we establish the asymptotic local minimax optimality of estimation based on finite markets. We then show that the theory can be used for conducting statistically valid A/B testing on auction platforms. Synthetic and semi-synthetic experiments verify the validity and practicality of our theory.
Paper Structure (78 sections, 50 theorems, 182 equations, 13 figures, 8 tables)

This paper contains 78 sections, 50 theorems, 182 equations, 13 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Contributions
  3. Statistical models for resource allocation systems and first-price pacing auction platforms.
  4. Convergence and inference results for LFM and FPPE.
  5. Statistically reliable A/B testing in resource allocation systems and FPPE platforms.
  6. Overview and Motivation of our Limit Market Model
  7. Related Works
  8. Notations
  9. Linear Fisher Market and First-Price Pacing Equilibrium
  10. Linear Fisher Markets (LFM)
  11. First-Price Pacing Equilibrium (FPPE)
  12. Differentiability Assumption
  13. Statistical Results for Linear Fisher Markets
  14. Basic Convergence Properties
  15. Asymptotics of Linear Fisher Market
  16. ...and 63 more sections

Key Result

Lemma 1

Suppose $H$ is twice continuously differentiable at the equilibrium pacing multiplier vector $\beta^*$. Then $\nabla H (\beta^*) = - {\delta^*}$, and $\nabla^{2} H(\beta^*) \beta^* = [b_1/\beta^*_1, \dots, b_n / \beta^*_n]^{\top}$.

Figures (13)

  • Figure 1: Items (orange) are assigned to buyers (blue). Left: a finite market. Right: a limit market.
  • Figure 2: Finite sample distributions of $\sqrt t (\beta^\gamma_i - {\beta^*_i})$ of 5 buyers in an FPPE. We see that for buyer 25, its finite-sample pacing multiplier is exactly 1 for most of the time. For buyer 21, its limit pacing multiplier is very close to 1 and so its distribution is not normal for small samples. For buyers 22 -- 24, their finite sample distribution is close to normal distributions. The full figure is in \ref{['fig:nonnormality_unif']}.
  • Figure 3: The fair notification allocation data. Left: original data with missing values. Right: simulated data.
  • Figure 4: Click-through rate (in basis points, i.e. 0.01%) distributions from logistic regression.
  • Figure 5: Click-through rate (in basis points, i.e. 0.01%) distributions from logistic regression.
  • ...and 8 more figures

Theorems & Definitions (103)

  • Example 1: Allocation of resources
  • Example 2: Fair notification allocation
  • Definition 1: Limit LFM
  • Definition 2: Finite LFM
  • Definition 3: Limit FPPE, gao2022infinite
  • Definition 4: Finite FPPE, conitzer2022pacing
  • Lemma 1
  • Theorem 1: Strong Consistency
  • Theorem 2
  • Theorem 3: Convergence of Approximate Market Equilibrium
  • ...and 93 more