Mechanism Design for Large Language Models
Paul Duetting, Vahab Mirrokni, Renato Paes Leme, Haifeng Xu, Song Zuo
TL;DR
The paper introduces a token-by-token auction framework for aggregating outputs from multiple LLM agents, modeling agent preferences over generated token distributions with robust partial orders. It proves that two minimal incentive properties imply monotone aggregation over token distributions and, under robust preferences, permit a second-price-like payment rule via stable sampling. The authors develop aggregation rules inspired by KL-divergence and RLHF losses, yielding linear and log-linear schemes, and demonstrate their behavior through prompt-tuned LLM experiments showing smooth transitions as bidder balance changes. This work provides a principled, distribution-focused mechanism design approach for influencing AI-generated content in ad-like applications, with practical viability demonstrated on real LLMs and clear guidance on design choices and limitations.
Abstract
We investigate auction mechanisms for AI-generated content, focusing on applications like ad creative generation. In our model, agents' preferences over stochastically generated content are encoded as large language models (LLMs). We propose an auction format that operates on a token-by-token basis, and allows LLM agents to influence content creation through single dimensional bids. We formulate two desirable incentive properties and prove their equivalence to a monotonicity condition on output aggregation. This equivalence enables a second-price rule design, even absent explicit agent valuation functions. Our design is supported by demonstrations on a publicly available LLM.