Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Auctions with LLM Summaries

Kumar Avinava Dubey, Zhe Feng, Rahul Kidambi, Aranyak Mehta, Di Wang

TL;DR

This work addresses ad auctions where the final display is an LLM-generated summary of multiple ads, complicating traditional welfare and incentive analysis. It develops a factorized framework comprising an Auction Module that outputs relative prominences $Prom \in [0,1]^n$ (sum ≤ 1), an LLM Module that converts Prom and ad features into a summary, and a pCTR Module that provides unbiased average CTR predictions $\text{pctr}_i(Prom,z)$, enabling end-to-end incentive compatibility. The paper establishes core principles such as faithfulness and monotonicity to ensure IC, proves a revelation-type universality result showing any IC scale-free meta-LLM mechanism can be realized by the factorized model, and analyzes a DWLS case where a generalized proportional allocation with $f(Prom_i)=Prom_i^\beta$ achieves welfare maximization with $\alpha=1/(1-\beta)$. Empirically, it demonstrates through synthetic data that the GPA+LLM approach yields higher welfare than baselines, especially under tight word budgets, illustrating the practical value of jointly optimizing prompts and allocations for flexible LLM-guided displays.

Abstract

We study an auction setting in which bidders bid for placement of their content within a summary generated by a large language model (LLM), e.g., an ad auction in which the display is a summary paragraph of multiple ads. This generalizes the classic ad settings such as position auctions to an LLM generated setting, which allows us to handle general display formats. We propose a novel factorized framework in which an auction module and an LLM module work together via a prediction model to provide welfare maximizing summary outputs in an incentive compatible manner. We provide a theoretical analysis of this framework and synthetic experiments to demonstrate the feasibility and validity of the system together with welfare comparisons.

Auctions with LLM Summaries

TL;DR

This work addresses ad auctions where the final display is an LLM-generated summary of multiple ads, complicating traditional welfare and incentive analysis. It develops a factorized framework comprising an Auction Module that outputs relative prominences (sum ≤ 1), an LLM Module that converts Prom and ad features into a summary, and a pCTR Module that provides unbiased average CTR predictions , enabling end-to-end incentive compatibility. The paper establishes core principles such as faithfulness and monotonicity to ensure IC, proves a revelation-type universality result showing any IC scale-free meta-LLM mechanism can be realized by the factorized model, and analyzes a DWLS case where a generalized proportional allocation with achieves welfare maximization with . Empirically, it demonstrates through synthetic data that the GPA+LLM approach yields higher welfare than baselines, especially under tight word budgets, illustrating the practical value of jointly optimizing prompts and allocations for flexible LLM-guided displays.

Abstract

We study an auction setting in which bidders bid for placement of their content within a summary generated by a large language model (LLM), e.g., an ad auction in which the display is a summary paragraph of multiple ads. This generalizes the classic ad settings such as position auctions to an LLM generated setting, which allows us to handle general display formats. We propose a novel factorized framework in which an auction module and an LLM module work together via a prediction model to provide welfare maximizing summary outputs in an incentive compatible manner. We provide a theoretical analysis of this framework and synthetic experiments to demonstrate the feasibility and validity of the system together with welfare comparisons.
Paper Structure (25 sections, 4 theorems, 9 equations, 2 figures, 2 tables)

This paper contains 25 sections, 4 theorems, 9 equations, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Our Contribution
  3. Factorized Model for Auctions with LLM Summaries.
  4. Related Work
  5. Model and Preliminaries
  6. Auction Module
  7. LLM Module
  8. pCTR Module
  9. The model is a generalization of Position Auctions
  10. Prominence-based Auction Design
  11. Incentive Compatibility
  12. Universality of the factorized model
  13. Welfare Maximization
  14. Case Study: Dynamic Word-length Summary (DWLS) Auctions
  15. Prompting Strategy
  16. ...and 10 more sections

Key Result

Proposition 3.3

Given a monotonic LLM (Def. def:monotone-llm) and an unbiased $\text{pctr}$ module (Def. ass:unbiased-pctr), a prominence-based auction $\mathcal{M} = (x, p)$ is incentive compatible (Def def:e2e-ic) if and only if

Figures (2)

  • Figure 1: Factorized model for Auctions with LLM Summaries.
  • Figure 2: Total welfare with different choices of total number of words for summarization.

Theorems & Definitions (13)

  • Definition 2.1
  • Definition 2.2: Faithfulness
  • Definition 2.4: Incentive Compatibility of the Factorized Model
  • Definition 3.1: monotone Allocation
  • Definition 3.2: monotonic LLM
  • Proposition 3.3: Incentive Compatibility of Prominence-based Auctions
  • Theorem 3.5
  • proof
  • Proposition 3.6
  • Definition 4.1: Generalized Proportional Allocation
  • ...and 3 more