Turnpike and dissipativity in generalized discrete-time stochastic linear-quadratic optimal control

TL;DR

The paper addresses long-horizon stochastic control by developing a time-varying $L^2$-dissipativity framework that replaces a deterministic steady state with a pair of stationary distributions. It derives that this stochastic dissipativity yields turnpike behavior for state distributions, moments, and even single trajectories in probability, and proves that the stationary distribution is characterized via a stationary optimization problem, with a linear-quadratic structure enabling explicit Riccati-based computation. The results unify pathwise, distributional, and moment-based turnpikes under a single dissipativity principle and are supported by numerical experiments for Gaussian and non-Gaussian disturbances. This contributes to the design and analysis of long-horizon stochastic controllers and informs model predictive control approaches under uncertainty.

Abstract

We investigate different turnpike phenomena of generalized discrete-time stochastic linear-quadratic optimal control problems. Our analysis is based on a novel strict dissipativity notion for such problems, in which a stationary stochastic process replaces the optimal steady state of the deterministic setting. We show that from this time-varying dissipativity notion, we can conclude turnpike behaviors concerning different objects like distributions, moments, or sample paths of the stochastic system and that the distributions of the stationary pair can be characterized by a stationary optimization problem. The analytical findings are illustrated by numerical simulations.

Figures (7)

• Figure 1: Mean-square distance between the stationary pair and the optimal solutions on different horizons. \newlabelfig:CSTRGaussianDiffExp0
• Figure 2: State and control trajectories for different initial conditions and horizons with one identical Gaussian noise realization in each row. \newlabelfig:CSTRGaussianSample2Rows0
• Figure 3: Wasserstein distance of order $2$ between the stationary distribution and the distribution of the optimal trajectories on different horizons $N$. \newlabelfig:CSTRGaussianWasserstein0
• Figure 4: Evolution of the expectations and variances for different horizons $N$ (solid blue) together with the corresponding moment of the stationary distribution (dashed red). \newlabelfig:CSTRGaussianMoments0
• Figure 5: State and control trajectories for different initial conditions and horizons with one identical gamma-distributed noise realization in each row.

Theorems & Definitions (43)

• Remark 2.1
• Remark 2.2
• Definition 2.3: Deterministic Strict Dissipativity
• Definition 2.4: Deterministic Turnpike
• Lemma 2.5: Gruene2013
• Definition 3.1
• Remark 3.2
• Remark 3.3
• Definition 3.4: Mean-square Dissipativity
• Remark 3.5