Relationships Between the Maximum Principle and Dynamic Programming for Infinite Dimensional Non-Markovian Stochastic Control Systems

TL;DR

The paper addresses the problem of relating Pontryagin's maximum principle (PMP) and dynamic programming (DPP) for stochastic control systems governed by stochastic evolution equations with random coefficients in infinite-dimensional, non-Markovian settings. It develops a DPP formulation for Problem (OP) using an essential infimum over adapted controls, and derives the associated stochastic Hamilton–Jacobi–Bellman (HJB) structure through the Hamiltonian $\mathbb{H}$, including a stochastic HJB equation and its deterministic counterpart when coefficients are non-random. In the smooth-value-function regime, explicit correspondences between the PMP adjoints and the DPP value function are established, while in the nonsmooth case nonsmooth analysis and relaxed transposition solutions are employed to obtain sample-wise PMP–DPP relations. The results are validated via illustrative examples and contribute a rigorous link between variational (PMP) and dynamic-programming (DPP) approaches for complex, infinite-dimensional stochastic control problems with non-Markovian dynamics.

Abstract

This paper investigates the relationship between Pontryagin's maximum principle and dynamic programming principle in the context of stochastic optimal control systems governed by stochastic evolution equations with random coefficients in separable Hilbert spaces. Our investigation proceeds through three contributions: (1). We first establish the formulation of the dynamic programming principle for this class of infinite-dimensional stochastic systems, subsequently deriving the associated stochastic Hamilton-Jacobi-Bellman equations that characterize the value function's evolution. (2). For systems with smooth value functions, we develop explicit correspondence relationships between Pontryagin's maximum principle and dynamic programming principle, elucidating their fundamental connections through precise mathematical characterizations. (3). In the more challenging non-smooth case, we employ tools in nonsmooth analysis and relaxed transposition solution techniques to uncover previously unknown sample-wise relationships between the two principles.

Theorems & Definitions (5)

• Definition 2.1
• Lemma 2.1
• Corollary 2.1
• Lemma 2.2
• proof : Proof of Lemma \ref{['lemma_esti_sde']}