Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Near Optimality of Discrete-Time Approximations for Controlled McKean-Vlasov Diffusions and Interacting Particle Systems

Somnath Pradhan, Serdar Yuksel

TL;DR

The paper tackles stochastic optimal control for McKean–Vlasov diffusions and interacting particle systems by developing a relaxed-control framework and discrete-time approximations. It shows existence of optimal relaxed policies, proves near-optimality of piecewise-constant controls, and establishes convergence of discrete-time value functions to their continuous-time counterparts with rates on both finite-horizon and discounted criteria. The authors extend these results to $N$-particle systems, demonstrating asymptotic optimality of discrete-time MV policies as $N\to\infty$ and $h\to0$, and introduce finite-model approximations via weak Feller regularity to obtain near-optimal solutions using quantized state spaces. Overall, the work provides a rigorous, numerically implementable pathway to approximate continuous-time MV control problems and large-scale interacting diffusions with explicit error bounds.

Abstract

We study stochastic optimal control problems for (possibly degenerate) McKean-Vlasov controlled diffusions and obtain discrete-time as well as finite interacting particle approximations. (i) Under mild assumptions, we first prove the existence of optimal relaxed controls by endowing the space of relaxed policies with a compact weak topology. (ii) Establishing continuity of the cost in control policy, we establish near-optimality of piecewise-constant strict policies, show that the discrete-time value functions (finite-horizon and discounted infinite-horizon) converge to their continuous-time counterparts as the timestep converges to zero, and that optimal discrete-time policies are near-optimal for the original continuous-time problem, where rates of convergence are also obtained. (iii) We then extend these approximation and near-optimality results to $N$-particle interacting systems under centralized or decentralized mean-field sharing information structure, proving that the discrete-time McKean-Vlasov policy is asymptotically optimal as $N\to \infty$ and the time step goes to zero. We thus develop an approximation of McKean-Vlasov optimal control problems via discrete-time McKean-Vlasov control problems (and associated numerical methods such as finite model approximation), and also show the near optimality of such approximate policy solutions for the $N$-agent interacting models under centralized and decentralized control.

Near Optimality of Discrete-Time Approximations for Controlled McKean-Vlasov Diffusions and Interacting Particle Systems

TL;DR

The paper tackles stochastic optimal control for McKean–Vlasov diffusions and interacting particle systems by developing a relaxed-control framework and discrete-time approximations. It shows existence of optimal relaxed policies, proves near-optimality of piecewise-constant controls, and establishes convergence of discrete-time value functions to their continuous-time counterparts with rates on both finite-horizon and discounted criteria. The authors extend these results to -particle systems, demonstrating asymptotic optimality of discrete-time MV policies as and , and introduce finite-model approximations via weak Feller regularity to obtain near-optimal solutions using quantized state spaces. Overall, the work provides a rigorous, numerically implementable pathway to approximate continuous-time MV control problems and large-scale interacting diffusions with explicit error bounds.

Abstract

We study stochastic optimal control problems for (possibly degenerate) McKean-Vlasov controlled diffusions and obtain discrete-time as well as finite interacting particle approximations. (i) Under mild assumptions, we first prove the existence of optimal relaxed controls by endowing the space of relaxed policies with a compact weak topology. (ii) Establishing continuity of the cost in control policy, we establish near-optimality of piecewise-constant strict policies, show that the discrete-time value functions (finite-horizon and discounted infinite-horizon) converge to their continuous-time counterparts as the timestep converges to zero, and that optimal discrete-time policies are near-optimal for the original continuous-time problem, where rates of convergence are also obtained. (iii) We then extend these approximation and near-optimality results to -particle interacting systems under centralized or decentralized mean-field sharing information structure, proving that the discrete-time McKean-Vlasov policy is asymptotically optimal as and the time step goes to zero. We thus develop an approximation of McKean-Vlasov optimal control problems via discrete-time McKean-Vlasov control problems (and associated numerical methods such as finite model approximation), and also show the near optimality of such approximate policy solutions for the -agent interacting models under centralized and decentralized control.
Paper Structure (15 sections, 20 theorems, 105 equations)

This paper contains 15 sections, 20 theorems, 105 equations.

Table of Contents

  1. Introduction
  2. Optimal Control of McKean-Vlasov Diffusions
  3. Continuity of Cost in Control Policy and Existence of Optimal Policies
  4. Topology on control policy
  5. Existence of an optimal policy
  6. Near optimality of piece-wise constant policies
  7. Discrete-Time Approximate McKean-Vlasov Model and Near Optimality
  8. Discrete-Time Approximate Model
  9. Near Optimality and Convergence Analysis
  10. Convergence for the finite horizon cost criterion
  11. Convergence for the discounted cost criterion
  12. Near-Optimality of the Discrete-Time Approximate McKean-Vlasov Solution for $N$-Particle Interacting Diffusions
  13. Finite Model Approximations: Near Optimality of Solutions under Weak Feller Regularity of the Discrete-Time Approximate Model
  14. Conclusion
  15. Acknowledgement

Key Result

Lemma 3.1

Let $m\in{\mathcal{M}}(\infty)$ be a relaxed control on probability space $(\Omega, {\mathfrak{F}}, {\mathscr{P}})$. Then there exists a sequence $m^{n}$ of non-anticipative piece-wise constant precise controls on $(\Omega, {\mathfrak{F}}, {\mathscr{P}})$ such that, for each $T>0$ and $f\in {\mathca

Theorems & Definitions (38)

  • Remark 2.1
  • Lemma 3.1
  • Theorem 3.2
  • proof
  • Remark 3.3
  • Remark 3.4
  • Lemma 3.5
  • proof
  • Remark 4.1
  • Theorem 4.2
  • ...and 28 more