Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Optimal Control of Stochastic Partial Differential Equations with Partial Observations: Stochastic Maximum Principles and Numerical Approximation

Yanzhao Cao, Hongjiang Qian, George Yin

TL;DR

This work tackles partially observed optimal control of semilinear SPDEs in a Hilbert space with correlated state-observation noise, and establishes Peng's stochastic maximum principle for nonconvex control domains via a measure change to $\mathbb{Q}$ and a second-order adjoint represented on $\Lambda^2$. It introduces a mollified second order adjoint state $P^\eta$ to handle the diagonal evaluation arising from quadratic variations, and proves the limit to a properly defined second-order adjoint. The authors then develop a practical numerical framework that combines finite element discretization, conditional stochastic gradient descent, and branching particle filtering to compute the optimal control using the SMP. Numerical experiments on a stochastic heat equation illustrate the method and demonstrate how the gradient-based updates converge to a partially observed optimal policy. The approach provides a scalable pathway for solving high-dimensional, partially observed stochastic control problems in SPDE settings with correlated noise and nonconvex actuation sets.

Abstract

This work establishes a general stochastic maximum principle for partially observed optimal control of semi-linear stochastic partial differential equations in a nonconvex control domain. The state evolves in a Hilbert space driven by a cylindrical Wiener process and finitely many Brownian motions, while observations are in an Euclidean space having correlated noise. For convex control domain and diffusion coefficients in the state being control-independent, numerical algorithms are developed to solve the partially observed optimal control problems using stochastic gradient descent algorithm combined with finite element approximations and the branching filtering algorithm. Numerical experiments are conducted for demonstration.

Optimal Control of Stochastic Partial Differential Equations with Partial Observations: Stochastic Maximum Principles and Numerical Approximation

TL;DR

This work tackles partially observed optimal control of semilinear SPDEs in a Hilbert space with correlated state-observation noise, and establishes Peng's stochastic maximum principle for nonconvex control domains via a measure change to and a second-order adjoint represented on . It introduces a mollified second order adjoint state to handle the diagonal evaluation arising from quadratic variations, and proves the limit to a properly defined second-order adjoint. The authors then develop a practical numerical framework that combines finite element discretization, conditional stochastic gradient descent, and branching particle filtering to compute the optimal control using the SMP. Numerical experiments on a stochastic heat equation illustrate the method and demonstrate how the gradient-based updates converge to a partially observed optimal policy. The approach provides a scalable pathway for solving high-dimensional, partially observed stochastic control problems in SPDE settings with correlated noise and nonconvex actuation sets.

Abstract

This work establishes a general stochastic maximum principle for partially observed optimal control of semi-linear stochastic partial differential equations in a nonconvex control domain. The state evolves in a Hilbert space driven by a cylindrical Wiener process and finitely many Brownian motions, while observations are in an Euclidean space having correlated noise. For convex control domain and diffusion coefficients in the state being control-independent, numerical algorithms are developed to solve the partially observed optimal control problems using stochastic gradient descent algorithm combined with finite element approximations and the branching filtering algorithm. Numerical experiments are conducted for demonstration.

Paper Structure

This paper contains 16 sections, 7 theorems, 101 equations, 2 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Notation and assumptions
  3. Main results
  4. Variational inequality
  5. Spike variation
  6. Derivation of variational inequality
  7. Adjoint representation of variational inequality
  8. Rewrite of ${\mathbb E}_1^{\mathbb Q}+{\mathbb E}_2^{\mathbb Q}$
  9. Mollified second order adjoint state
  10. The limit of the mollified second order adjoint equation
  11. Proof of Theorem \ref{['thm:Peng-smp']}
  12. Numerical algorithm for solving partially observed optimal control
  13. Conditional SGD algorithm
  14. The branching algorithm
  15. Numerical experiments
  16. ...and 1 more sections

Key Result

Theorem 2.2

Let $(\widehat{x}, \widehat{u})$ be an optimal pair of the partially observed optimal control problem new-spde--contr-prob. Suppose that the control domain is convex. Then there exist $(z,z_1,z_2)$ and $(p,q_1,q_2)$ satisfying z and pq, respectively, such that for any $v\in \mathcal{U}_{\text{ad}}$ where $\{e_i\}$ is an orthonormal basis of $L^2(\Lambda)$.

Figures (2)

  • Figure 1: Sample path of stochastic heat equations without control.
  • Figure 2: Sample path of approximated partially observed optimal control and the corresponding sample path of optimal controlled state.

Theorems & Definitions (14)

  • Theorem 2.2: Pontryagin's maximum principle
  • Theorem 2.3: Peng's maximum principle
  • Lemma 3.1
  • proof
  • Lemma 3.2
  • proof
  • Proposition 4.1
  • proof
  • Theorem 4.2
  • proof
  • ...and 4 more