Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A dynamic programming principle for multiperiod control problems with bicausal constraints

Ruslan Mirmominov, Johannes Wiesel

Abstract

We consider multiperiod stochastic control problems with non-parametric uncertainty on the underlying probabilistic model. We derive a new metric on the space of probability measures, called the adapted $(p, \infty)$--Wasserstein distance $\mathcal{AW}_p^\infty$ with the following properties: (1) the adapted $(p, \infty)$--Wasserstein distance generates a topology that guarantees continuity of stochastic control problems and (2) the corresponding $\mathcal{AW}_p^\infty$-distributionally robust optimization (DRO) problem can be computed via a dynamic programming principle involving one-step Wasserstein-DRO problems. If the cost function is semi-separable, then we further show that a minimax theorem holds, even though balls with respect to $\mathcal{AW}_p^\infty$ are neither convex nor compact in general. We also derive first-order sensitivity results.

A dynamic programming principle for multiperiod control problems with bicausal constraints

Abstract

We consider multiperiod stochastic control problems with non-parametric uncertainty on the underlying probabilistic model. We derive a new metric on the space of probability measures, called the adapted --Wasserstein distance with the following properties: (1) the adapted --Wasserstein distance generates a topology that guarantees continuity of stochastic control problems and (2) the corresponding -distributionally robust optimization (DRO) problem can be computed via a dynamic programming principle involving one-step Wasserstein-DRO problems. If the cost function is semi-separable, then we further show that a minimax theorem holds, even though balls with respect to are neither convex nor compact in general. We also derive first-order sensitivity results.

Paper Structure

This paper contains 15 sections, 38 theorems, 300 equations.

Table of Contents

  1. Introduction
  2. Optimal transport, bicausality and the adapted Wasserstein distance
  3. The adapted $(p,\infty)$--Wasserstein distance $\mathcal{AW}_p^\infty$
  4. A dynamic programming principle for $\mathcal{AW}_p^\infty$-DRO problems
  5. Properties of $B_\delta(\mu)$
  6. Main results
  7. First-order sensitivity for $\mathcal{AW}_p^\infty$-DRO problems
  8. Sensitivity for $\mathcal{AW}_p^\infty$-DRO problems with martingale constraint
  9. Remaining proofs
  10. Proofs of Lemma \ref{['lem:adap_new']} and Lemma \ref{['lem:easy']}
  11. Auxiliary results for the proof of Proposition \ref{['prop:equivalence']}
  12. Auxiliary results for the proofs in Sections \ref{['sec:dpp']} and \ref{['sec:sens']}
  13. Proof of Theorem \ref{['equiv:basic']}
  14. Proofs of Lemma \ref{['lem:cost_to_go_regularity']}, Theorem \ref{['equiv:control_open']}, Corollary \ref{['cor:linear']} and Theorem \ref{['minimax']}
  15. Proof of Theorem \ref{['dro:control_sensitivity']} and Corollary \ref{['dro:control_martingale_sensitivity']}

Key Result

Lemma 3.2

For $t=1, \dots, N$ and $\mu, \nu \in \mathcal{P}_p(\mathbb{R}^{N})$ define $A_{N,p}:=0$ and Then we have In particular

Theorems & Definitions (99)

  • Definition 2.1
  • Definition 2.2
  • Example 2.3
  • Definition 3.1
  • Lemma 3.2: DPP formulation for $\mathcal{AW}_p^\infty$
  • Remark 3.3
  • Lemma 3.4
  • Example 3.5
  • Proposition 3.6: Equivalence of $\mathcal{AW}_p$ and $\mathcal{AW}_p^\infty$
  • proof
  • ...and 89 more