On the relationship between stochastic turnpike and dissipativity notions

TL;DR

This work develops a comprehensive link between stochastic dissipativity and turnpike behavior for discrete-time nonlinear OCPs. By formulating two dissipativity notions—distributional dissipativity on probability measures and $L^r$ dissipativity on random variables—and four corresponding turnpike types, the authors establish a hierarchy of results showing that strict $L^r$ dissipativity implies $L^s$ turnpikes for $s\le r$, and also implies strict distributional dissipativity and distributional turnpikes. They further show that optimal long-run behavior is characterized by an optimal stationary distribution and policy, with uniqueness under strict distributional dissipativity, and provide a nonlinear example to illustrate the theory. The framework integrates Markov policies, transition operators, and metric-based distance notions (including Wasserstein and Lévy–Prokhorov) to connect pathwise, distributional, and moment-based perspectives. These insights have direct implications for stochastic MPC design, stability analysis, and robustness in stochastic environments.

Abstract

In this paper, we introduce and study different dissipativity notions and different turnpike properties for discrete-time stochastic nonlinear optimal control problems. The proposed stochastic dissipativity notions extend the classic notion of Jan C. Willems to $L^r$ random variables and to probability measures. Our stochastic turnpike properties range from a formulation for random variables via turnpike phenomena in probability and in probability measures to the turnpike property for the moments. Moreover, we investigate how different metrics (such as Wasserstein or Lévy-Prokhorov) can be leveraged in the analysis. Our results are built upon stationarity concepts in distribution and in random variables and on the formulation of the stochastic optimal control problem as a finite-horizon Markov decision process. We investigate how the proposed dissipativity notions connect to the various stochastic turnpike properties and we work out the link between different forms of dissipativity.

Figures (2)

• Figure 1: Schematic sketch of the main results.
• Figure 2: Evolution of all realization paths for the optimal trajectories (left) and controls (right) with $N=3$ in red, $N=5$ in green, $N=7$ in blue, $N=10$ in orange, and $N=12$ in purple, as well as for the optimal stationary process (grey)

Theorems & Definitions (34)

• Remark II.1
• Definition II.2: Stationary policy and distribution
• Definition II.3: Stationary stochastic processes
• Definition II.4: Distributional dissipativity
• Definition II.5: $L^r$ dissipativity in random variables
• Remark II.6: The role of $L^r$ in \ref{['eq:DissiIneqLp']}
• Remark II.7: Non-Markovian extensions
• Definition II.8: Stochastic turnpike properties
• Definition II.9
• Remark II.10