Infinite Horizon Average Cost Optimality Criteria for Mean-Field Control
Erhan Bayraktar, Ali D. Kara
TL;DR
This work develops a rigorous framework for discrete-time mean-field control under the infinite-horizon average-cost criterion, addressing both finite-population and infinite-population settings. By formulating a measure-valued MDP and proving the existence of solutions to the average-cost optimality equation (ACOE) under two regimes—minorization with geometric ergodicity and Lipschitz continuity—the paper establishes optimal stationary policies in finite populations and, via a vanishing-discount approach, analogous results for the infinite-population limit. It then shows that finite-population values converge to the infinite-population optimum as the number of agents grows, and that accumulation points of finite-population policies align with infinite-population optimal flows; moreover, symmetric policies derived from the infinite-population problem are near-optimal for large N, bridging finite and infinite analyses. These results provide a principled way to design near-optimal, scalable policies for large multi-agent systems and offer insights into when symmetric, decentralized strategies suffice. The work thus advances mean-field control by linking ergodic and continuity-based existence results with finite-to-infinite convergence and practical near-optimality guarantees.
Abstract
We study mean-field control problems in discrete-time under the infinite horizon average cost optimality criteria. We focus on both the finite population and the infinite population setups. We show the existence of a solution to the average cost optimality equation (ACOE) and the existence of optimal stationary Markov policies for finite population problems under (i) a minorization condition that provides geometric ergodicity on the collective state process of the agents, and (ii) under standard Lipschitz continuity assumptions on the stage-wise cost and transition function of the agents when the Lipschitz constant of the transition function satisfies a certain bound. For the infinite population problem, we establish the existence of a solution to the ACOE, and the existence of optimal policies under the continuity assumptions on the cost and the transition functions. Finally, we relate the finite population and infinite population control problems: (i) we prove that the optimal value of the finite population problem converges to the optimal value of the infinite population problem as the number of agents grows to infinity; (ii) we show that the accumulation points of the finite population optimal solution corresponds to an optimal solution for the infinite population problem, and finally (iii), we show that one can use the solution of the infinite population problem for the finite population problem symmetrically across the agents to achieve near optimal performance when the population is sufficiently large.