Truthful Reverse Auctions for Adaptive Selection via Contextual Multi-Armed Bandits
Pronoy Patra, Sankarshan Damle, Manisha Padala, Sujit Gujar
TL;DR
This work develops a truthful, reverse-contextual multi-armed bandit framework for adaptive LLM model selection where a user elicits private costs from providers. The authors introduce ROSA, a reverse self-resampling procedure, to preserve Bayesian incentive compatibility in reverse auctions, and couple it with TRCM-UCB_OPT, a contextual MAB learner that maintains allocation monotonicity. They prove ex-post truthfulness and no-loss guarantees for the mechanism and establish sublinear regret (with $O(\sqrt{T})$) in stochastic settings. The approach unifies mechanism design with online learning to enable efficient, query-aware, provider-optimal LLM allocation, validated by simulations under Gaussian and Exponential reward models. This framework lays the groundwork for truthful, incentive-compatible marketplace designs in multi-model AI ecosystems.
Abstract
We study the problem of selecting large language models (LLMs) for user queries in settings where multiple LLM providers submit the cost of solving a query. From the users' perspective, choosing an optimal model is a sequential, query-dependent decision problem: high-capacity models offer more reliable outputs but are costlier, while lightweight models are faster and cheaper. We formalize this interaction as a reverse auction design problem with contextual online learning, where the user adaptively discovers which model performs best while eliciting costs from competing LLM providers. Existing multi-armed bandit (MAB) mechanisms focus on forward auctions and social welfare, leaving open the challenges of reverse auctions, provider-optimal outcomes, and contextual adaptation. We address these gaps by designing a resampling-based procedure that generalizes truthful forward MAB mechanisms to reverse auctions and prove that any monotone allocation rule with this procedure is truthful. Using this, we propose a contextual MAB algorithm that learns query-dependent model quality with sublinear regret. Our framework unifies mechanism design and adaptive learning, enabling efficient, truthful, and query-aware LLM selection.