Steering No-Regret Agents in MFGs under Model Uncertainty
Leo Widmer, Jiawei Huang, Niao He
TL;DR
The paper addresses steering no-regret agents in large-population mean-field games under model uncertainty, focusing on density-independent transitions. It develops an optimistic, model-free steering framework that steers agents toward target policies while ensuring sub-linear steering gaps and steering costs, even when intrinsic rewards are unknown or non-zero. Core contributions include a formal steering formulation for MFGs, a policy-incentivization approach that works without known dynamics, and sub-linear regret guarantees under zero intrinsic reward, plus a pessimism-based reward estimation strategy to handle non-zero unknown intrinsic rewards. The results enable scalable, principled guidance of learning agents in vast populations with uncertain models, with theoretical guarantees driven by online learning and Eluder-dimension tools. The framework has potential for practical deployment in economics, traffic, and large-scale multi-agent systems where model uncertainty is intrinsic and exploration is necessary.
Abstract
Incentive design is a popular framework for guiding agents' learning dynamics towards desired outcomes by providing additional payments beyond intrinsic rewards. However, most existing works focus on a finite, small set of agents or assume complete knowledge of the game, limiting their applicability to real-world scenarios involving large populations and model uncertainty. To address this gap, we study the design of steering rewards in Mean-Field Games (MFGs) with density-independent transitions, where both the transition dynamics and intrinsic reward functions are unknown. This setting presents non-trivial challenges, as the mediator must incentivize the agents to explore for its model learning under uncertainty, while simultaneously steer them to converge to desired behaviors without incurring excessive incentive payments. Assuming agents exhibit no(-adaptive) regret behaviors, we contribute novel optimistic exploration algorithms. Theoretically, we establish sub-linear regret guarantees for the cumulative gaps between the agents' behaviors and the desired ones. In terms of the steering cost, we demonstrate that our total incentive payments incur only sub-linear excess, competing with a baseline steering strategy that stabilizes the target policy as an equilibrium. Our work presents an effective framework for steering agents behaviors in large-population systems under uncertainty.