Online Learning for Dynamic Vickrey-Clarke-Groves Mechanism in Unknown Environments
Vincent Leon, S. Rasoul Etesami
TL;DR
This work formulates online dynamic mechanism design for sequential auctions as an unknown-environment, infinite-horizon average-reward MDP and extends VCG to a dynamic setting. It introduces IHMDP-VCG, an episode-based online learning algorithm that uses occupancy-measure LPs, confidence sets, and phased exploration to approximate the offline dynamic VCG mechanism. Theoretical guarantees show ε-approximate efficiency, approximate truthfulness, and approximate IR, with regrets scaling as Ō(nεT + T^{2/3}) under high probability, and Φ-structured bounds when the horizon is known. This approach provides a principled, learnable framework for dynamic, incentive-compatible optimization in evolving markets with unknown transition dynamics, with potential impact on automated mechanism design in repeated auctions and adaptive marketplaces.
Abstract
We consider the problem of online dynamic mechanism design for sequential auctions in unknown environments, where the underlying market and, thus, the bidders' values vary over time as interactions between the seller and the bidders progress. We model the sequential auctions as an infinite-horizon average-reward Markov decision process (MDP). In each round, the seller determines an allocation and sets a payment for each bidder, while each bidder receives a private reward and submits a sealed bid to the seller. The state, which represents the underlying market, evolves according to an unknown transition kernel and the seller's allocation policy without episodic resets. We first extend the Vickrey-Clarke-Groves (VCG) mechanism to sequential auctions, thereby obtaining a dynamic counterpart that preserves the desired properties: efficiency, truthfulness, and individual rationality. We then focus on the online setting and develop a reinforcement learning algorithm for the seller to learn the underlying MDP and implement a mechanism that closely resembles the dynamic VCG mechanism. We show that the learned mechanism approximately satisfies efficiency, truthfulness, and individual rationality and achieves guaranteed performance in terms of various notions of regret.