Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Controlled rough SDEs, pathwise stochastic control and dynamic programming principles

Peter K. Friz, Khoa Lê, Huilin Zhang

TL;DR

This work develops a unifying rough-path framework for stochastic optimal control of RSDEs by replacing conditioning noise with a deterministic rough input $\mathbf{X}$ and studying the resulting rough value function $\mathcal{V}(s,y;\mathbf{X})$. It proves regularity and a rough dynamic programming principle, and provides a measurable-selection approach to connect RSDEs with doubly stochastic SDEs under conditioning. Through two-stage randomization, it shows that different admissible control classes yield the same stochastic value function in a precise sense and derives a stochastic DPP for the randomized problems. The results extend to Brownian noise, where causal controls recover classical Itô SDEs and reveal a clean connection to doubly stochastic SDEs, thereby unifying pathwise and probabilistic control perspectives without requiring conditioning noise statistics. Overall, the paper offers a coherent, robust treatment of pathwise stochastic control under rough inputs with implications for filtering, SPDEs, and reinforcement learning.

Abstract

We study stochastic optimal control of rough stochastic differential equations (RSDEs). This is in the spirit of the pathwise control problem (Lions--Souganidis 1998, Buckdahn--Ma 2007; also Davis--Burstein 1992), with renewed interest and recent works drawing motivation from filtering, SPDEs, and reinforcement learning. Results include regularity of rough value functions, validity of a rough dynamic programming principles and new rough stability results for HJB equations, removing excessive regularity demands previously imposed by flow transformation methods. Measurable selection is used to relate RSDEs to "doubly stochastic" SDEs under conditioning. In contrast to previous works, Brownian statistics for the to-be-conditioned-on noise are not required, aligned with the "pathwise" intuition that these should not matter upon conditioning. Depending on the chosen class of admissible controls, the involved processes may also be anticipating. The resulting stochastic value functions coincide in great generality for different classes of controls. RSDE theory offers a powerful and unified perspective on this problem class.

Controlled rough SDEs, pathwise stochastic control and dynamic programming principles

TL;DR

This work develops a unifying rough-path framework for stochastic optimal control of RSDEs by replacing conditioning noise with a deterministic rough input and studying the resulting rough value function . It proves regularity and a rough dynamic programming principle, and provides a measurable-selection approach to connect RSDEs with doubly stochastic SDEs under conditioning. Through two-stage randomization, it shows that different admissible control classes yield the same stochastic value function in a precise sense and derives a stochastic DPP for the randomized problems. The results extend to Brownian noise, where causal controls recover classical Itô SDEs and reveal a clean connection to doubly stochastic SDEs, thereby unifying pathwise and probabilistic control perspectives without requiring conditioning noise statistics. Overall, the paper offers a coherent, robust treatment of pathwise stochastic control under rough inputs with implications for filtering, SPDEs, and reinforcement learning.

Abstract

We study stochastic optimal control of rough stochastic differential equations (RSDEs). This is in the spirit of the pathwise control problem (Lions--Souganidis 1998, Buckdahn--Ma 2007; also Davis--Burstein 1992), with renewed interest and recent works drawing motivation from filtering, SPDEs, and reinforcement learning. Results include regularity of rough value functions, validity of a rough dynamic programming principles and new rough stability results for HJB equations, removing excessive regularity demands previously imposed by flow transformation methods. Measurable selection is used to relate RSDEs to "doubly stochastic" SDEs under conditioning. In contrast to previous works, Brownian statistics for the to-be-conditioned-on noise are not required, aligned with the "pathwise" intuition that these should not matter upon conditioning. Depending on the chosen class of admissible controls, the involved processes may also be anticipating. The resulting stochastic value functions coincide in great generality for different classes of controls. RSDE theory offers a powerful and unified perspective on this problem class.

Paper Structure

This paper contains 20 sections, 25 theorems, 148 equations.

Table of Contents

  1. Introduction
  2. Preliminaries and Notation
  3. Generalities
  4. Rough paths
  5. Function spaces
  6. Snapshots of RDEs and RSDE
  7. Rough paths and its integration
  8. Rough stochastic differential equations (RSDEs)
  9. Measurable selection for RSDEs
  10. Rough stochastic control
  11. Setup and assumptions
  12. Regularity of the rough value function
  13. Rough dynamical programming principle
  14. Explicit example
  15. Joint measurability of $\varepsilon$-minimizers
  16. ...and 5 more sections

Key Result

Proposition 3.4

Suppose that $(Z,Z')$ belongs to $\mathbf{D}_X^{\kappa,\kappa'} L_{p,q}$ with $\alpha+\kappa>1/2$, $\alpha+\min(\alpha,\kappa)+\kappa'>1$. Then the Riemann sums converge uniformly in time $t\in[0,T]$ in $L_p$, as $|{\mathcal{P}}|$ goes to 0, to a continuous adapted process, denoted by $\int_0^\cdot (Z,Z')d\mathbf{X}$.

Theorems & Definitions (63)

  • Remark 3.1
  • Remark 3.2
  • Definition 3.3: FHL21
  • Proposition 3.4
  • Definition 3.5: FHL21
  • Definition 3.6: FHL21
  • Definition 3.7: Stochastic controlled vector fields, FHL21
  • Definition 3.8: Controlled vector fields
  • Theorem 3.9: see FHL21
  • Theorem 3.10: see FHL21
  • ...and 53 more