Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Dynamic programming and dimensionality in convex stochastic optimization and control

Teemu Pennanen, Ari-Pekka Perkkiö

TL;DR

This work addresses high-dimensional stochastic optimization under a decision-hazard-decision information structure, where cost-to-go functions can be reduced in dimension. It develops a convex normal integrand DP framework that yields generalized Bellman equations for the cost-to-go and establishes existence results without requiring compactness. The key contributions include optimality conditions (verification-style), existence theorems, and explicit dimensionality reductions under Markov structure, together with an extension of stochastic dual dynamic programming to the DH-D setting. The results offer scalable DP methods applicable to energy systems and other two-time-scale stochastic optimization problems, by keeping the DP in the original state space and leveraging conditional independence and Markov properties.

Abstract

This paper studies stochastic optimization problems and associated Bellman equations in formats that allow for reduced dimensionality of the cost-to-go functions. In particular, we study stochastic control problems in the ``decision-hazard-decision'' form where at each stage, the system state is controlled both by predictable as well as adapted controls. Such an information structure may result in a lower dimensional system state than what is required in more traditional ``decision-hazard'' or ``hazard-decision'' formulations. The dimension is critical for the complexity of numerical dynamic programming algorithms and, in particular, for cutting plane schemes such as the stochastic dual dynamic programming algorithm. Our main result characterizes optimal solutions and optimum values in terms of solutions to generalized Bellman equations. Existence of solutions to the Bellman equations is established under general conditions that do not require compactness. We allow for general randomness but show that, in the Markovian case, the dimensionality of the Bellman equations reduces with respect to randomness just like in more traditional control formulations.

Dynamic programming and dimensionality in convex stochastic optimization and control

TL;DR

This work addresses high-dimensional stochastic optimization under a decision-hazard-decision information structure, where cost-to-go functions can be reduced in dimension. It develops a convex normal integrand DP framework that yields generalized Bellman equations for the cost-to-go and establishes existence results without requiring compactness. The key contributions include optimality conditions (verification-style), existence theorems, and explicit dimensionality reductions under Markov structure, together with an extension of stochastic dual dynamic programming to the DH-D setting. The results offer scalable DP methods applicable to energy systems and other two-time-scale stochastic optimization problems, by keeping the DP in the original state space and leveraging conditional independence and Markov properties.

Abstract

This paper studies stochastic optimization problems and associated Bellman equations in formats that allow for reduced dimensionality of the cost-to-go functions. In particular, we study stochastic control problems in the ``decision-hazard-decision'' form where at each stage, the system state is controlled both by predictable as well as adapted controls. Such an information structure may result in a lower dimensional system state than what is required in more traditional ``decision-hazard'' or ``hazard-decision'' formulations. The dimension is critical for the complexity of numerical dynamic programming algorithms and, in particular, for cutting plane schemes such as the stochastic dual dynamic programming algorithm. Our main result characterizes optimal solutions and optimum values in terms of solutions to generalized Bellman equations. Existence of solutions to the Bellman equations is established under general conditions that do not require compactness. We allow for general randomness but show that, in the Markovian case, the dimensionality of the Bellman equations reduces with respect to randomness just like in more traditional control formulations.
Paper Structure (6 sections, 9 theorems, 47 equations)

This paper contains 6 sections, 9 theorems, 47 equations.

Table of Contents

  1. Introduction
  2. Dynamic programming in optimal control
  3. Dynamic programming in problems of Lagrange
  4. Dimensionality of the Bellman equations
  5. Proofs of Theorems \ref{['opoc']}, \ref{['existoc']} and \ref{['dimxioc']}
  6. Stochastic dual dynamic programming

Key Result

Theorem 1

If $(J_t,\tilde{Q}_t)_{t=0}^T$ is a lower bounded convex normal solution of be, then the optimum value of oc equals that of for every $s=0,\ldots,T$, and an $(\bar{X},\bar{U}^b,\bar{U}^a)\in{\cal N}$ solves oc if and only if for all $t=0,\ldots,T-1$ almost surely.

Theorems & Definitions (19)

  • Theorem 1: Optimality principle
  • Theorem 2: Existence of solutions
  • Remark 3
  • Theorem 4: Dimensionality with respect to scenarios
  • Remark 5
  • Theorem 6: Optimality principle
  • proof
  • Theorem 7: Existence of solutions
  • proof
  • Theorem 8: Dimensionality with respect to scenarios
  • ...and 9 more