Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The turnpike control in stochastic multi-agent dynamics: a discrete-time approach with exponential integrators

Fabio Cassini, Chiara Segala

TL;DR

This work addresses the turnpike phenomenon in stochastic, discrete-time optimal control of interacting agents, showing that the optimal trajectory stays near the static turnpike solution for most of a long horizon under dissipativity and cheap-control conditions. The authors introduce exponential Rosenbrock time discretization to effectively handle stiffness in the agent interactions and provide a formal discrete-time turnpike theory, including interior decay and an explicit turnpike-time estimate. Numerical experiments with $N=100$ agents and stochastic perturbations demonstrate superior stability of exponential integrators over explicit schemes and validate the persistence of turnpike control in the stochastic setting. The results offer a scalable framework for long-horizon, uncertain multi-agent control with practical implications for reducing computational effort in large-scale systems while guaranteeing convergence toward a desired state $\bar{x}$.

Abstract

In this manuscript, we study the turnpike property in stochastic discrete-time optimal control problems for interacting agents. Extending previous deterministic results, we show that the turnpike effect persists in the presence of noise under suitable dissipativity and controllability conditions. To handle the possible stiffness in the system dynamics, we employ for the time discretization, integrators of exponential type. Numerical experiments validate our findings, demonstrating the advantages of exponential integrators over standard explicit schemes and confirming the effectiveness of the turnpike control even in the stochastic setting.

The turnpike control in stochastic multi-agent dynamics: a discrete-time approach with exponential integrators

TL;DR

This work addresses the turnpike phenomenon in stochastic, discrete-time optimal control of interacting agents, showing that the optimal trajectory stays near the static turnpike solution for most of a long horizon under dissipativity and cheap-control conditions. The authors introduce exponential Rosenbrock time discretization to effectively handle stiffness in the agent interactions and provide a formal discrete-time turnpike theory, including interior decay and an explicit turnpike-time estimate. Numerical experiments with agents and stochastic perturbations demonstrate superior stability of exponential integrators over explicit schemes and validate the persistence of turnpike control in the stochastic setting. The results offer a scalable framework for long-horizon, uncertain multi-agent control with practical implications for reducing computational effort in large-scale systems while guaranteeing convergence toward a desired state .

Abstract

In this manuscript, we study the turnpike property in stochastic discrete-time optimal control problems for interacting agents. Extending previous deterministic results, we show that the turnpike effect persists in the presence of noise under suitable dissipativity and controllability conditions. To handle the possible stiffness in the system dynamics, we employ for the time discretization, integrators of exponential type. Numerical experiments validate our findings, demonstrating the advantages of exponential integrators over standard explicit schemes and confirming the effectiveness of the turnpike control even in the stochastic setting.

Paper Structure

This paper contains 13 sections, 3 theorems, 70 equations, 7 figures.

Table of Contents

  1. Introduction
  2. Stochastic control problem
  3. Static control problem
  4. Time discretization using exponential integrators
  5. The turnpike property for the time-discrete stochastic control problem
  6. The strict dissipativity property
  7. The cheap control condition
  8. The turnpike property with interior decay
  9. Estimation of the turnpike time tbar
  10. Numerical experiments
  11. Examples with no control in the dynamics
  12. Tests with turnpike control
  13. Conclusions

Key Result

Lemma 4.1

Let the control penalization parameter $\gamma \in (0,1]$. The time-discrete optimal control problem $\mathcal{Q}(\tau, t_0, T, \boldsymbol{x}^0)$ is strictly dissipative with respect to the supply rate function $\eta(\boldsymbol{x}^n, \boldsymbol{u}^n)$. That is, there exists a storage function $S:

Figures (7)

  • Figure 1: Mean value of the computed trajectories when there is no control in the dynamics and $\epsilon=5\cdot 10^{-2}$. The number of time steps is $m=25$. Top plot: Euler--Maruyama method (EM). Bottom plot: stochastic exponential Rosenbrock--Euler method (SERB).
  • Figure 2: Mean value of the computed trajectories when there is no control in the dynamics and $\epsilon=5\cdot 10^{-2}$. Top: Euler--Maruyama method (EM) with $m=150$ time steps. Bottom: Euler--Maruyama method (EM) with $m=1500$ time steps.
  • Figure 3: Mean value of the computed trajectories when there is no control in the dynamics and $\epsilon=1$. The number of time steps is $m=50$. Top: Euler--Maruyama method (EM). Bottom: stochastic exponential Rosenbrock--Euler method (SERB).
  • Figure 4: Computed trajectories when there is no control in the dynamics and $\epsilon=5\cdot 10^{-2}$. The number of time steps is $m=25$. Top plot: explicit Euler method (EE). Center plot: Heun's method (RK2). Bottom plot: exponential Rosenbrock--Euler method (ERB).
  • Figure 5: Computed trajectories when there is no control in the dynamics and $\epsilon=5\cdot 10^{-2}$. The number of time steps is $m=1500$. Top plot: explicit Euler method (EE). Bottom plot: Heun's method (RK2).
  • ...and 2 more figures

Theorems & Definitions (7)

  • Remark 1
  • Lemma 4.1
  • proof
  • Lemma 4.3
  • proof
  • Theorem 4.4
  • proof