Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Numerical approximations for partially observed optimal control of stochastic partial differential equations

Feng Bao, Yanzhao Cao, Hongjiang Qian

TL;DR

This work tackles the challenge of optimally controlling stochastic partial differential equations with partial observations. It develops a stochastic maximum principle via a measure transformation to handle partial information, yielding a Hamiltonian that involves adjoint BSPDE/BSDE components. The authors then design a fully implementable numerical framework that couples finite-element spatial discretization, implicit-time stepping for forward-backward SPDEs, particle filtering for state estimation, and stochastic-gradient-descent updates guided by the SMP. The approach enables solving partially observed SPDE control problems in high-dimensional settings and is demonstrated through numerical experiments, highlighting its potential for applications in distributed parameter systems. Overall, the paper advances the computational toolkit for partially observed SPDE control by marrying analytical SMP with practical, scalable algorithms.

Abstract

In this paper, we study numerical approximations for optimal control of a class of stochastic partial differential equations with partial observations. The system state evolves in a Hilbert space, whereas observations are given in finite-dimensional space $\rr^d$. We begin by establishing stochastic maximum principles (SMPs) for such problems, where the system state is driven by a cylindrical Wiener process. The corresponding adjoint equations are characterized by backward stochastic partial differential equations. We then develop numerical algorithms to solve the partially observed optimal control. Our approach combines the stochastic gradient descent method, guided by the SMP, with a particle filtering algorithm to estimate the conditional distributions of the state of the system. Finally, we demonstrate the effectiveness of our proposed algorithm through numerical experiments.

Numerical approximations for partially observed optimal control of stochastic partial differential equations

TL;DR

This work tackles the challenge of optimally controlling stochastic partial differential equations with partial observations. It develops a stochastic maximum principle via a measure transformation to handle partial information, yielding a Hamiltonian that involves adjoint BSPDE/BSDE components. The authors then design a fully implementable numerical framework that couples finite-element spatial discretization, implicit-time stepping for forward-backward SPDEs, particle filtering for state estimation, and stochastic-gradient-descent updates guided by the SMP. The approach enables solving partially observed SPDE control problems in high-dimensional settings and is demonstrated through numerical experiments, highlighting its potential for applications in distributed parameter systems. Overall, the paper advances the computational toolkit for partially observed SPDE control by marrying analytical SMP with practical, scalable algorithms.

Abstract

In this paper, we study numerical approximations for optimal control of a class of stochastic partial differential equations with partial observations. The system state evolves in a Hilbert space, whereas observations are given in finite-dimensional space . We begin by establishing stochastic maximum principles (SMPs) for such problems, where the system state is driven by a cylindrical Wiener process. The corresponding adjoint equations are characterized by backward stochastic partial differential equations. We then develop numerical algorithms to solve the partially observed optimal control. Our approach combines the stochastic gradient descent method, guided by the SMP, with a particle filtering algorithm to estimate the conditional distributions of the state of the system. Finally, we demonstrate the effectiveness of our proposed algorithm through numerical experiments.

Paper Structure

This paper contains 14 sections, 3 theorems, 95 equations, 4 figures.

Table of Contents

  1. Introduction
  2. Preliminaries and main results
  3. Notation and assumptions
  4. A reference example
  5. Stochastic maximum principle
  6. Spatial-time discretization of forward-backward SPDEs
  7. Spatial discretization for FB-SPDEs
  8. Time discretization for spatial-discretized FBSPDE
  9. Numerical algorithm for solving partially observed optimal control
  10. Update of the control
  11. Particle filtering for computing conditional distribution
  12. Stochastic gradient descent algorithm
  13. Numerical Examples
  14. Concluding remarks

Key Result

Theorem 2.3

Let assumptions (H1)-(H4) hold and assume that $u^\star$ is an optimal control and $X^\star$ is the corresponding optimal state. Then for any $v\in \mathcal{U}_{\text{ad}}$, it is necessary to satisfy where $\{Q^\star,q_1^\star,q_2^\star \}$ and $\{C^\star, c_1^\star, c_2^\star\}$ are solutions of BSPDE with $X(\cdot),u(\cdot)$ replaced by $X^\star, u^\star$. Define $\nabla_u H(u^\star):= \nabla_

Figures (4)

  • Figure 1: Sample path of stochastic heat equation without control.
  • Figure 2: Sample path of approximated partially observed optimal control and the corresponding sample path of optimal controlled state solution.
  • Figure 3: Sample path of reference state and uncontrolled stochastic Nagumo equations.
  • Figure 4: Sample of approximated partially observed optimal control and the corresponding sample path of optimal controlled stochastic Nagumo equations.

Theorems & Definitions (10)

  • Remark 2.1
  • Theorem 2.3
  • Lemma 2.4
  • proof
  • proof : Proof of Theorem \ref{['thm:SMP']}
  • Theorem 3.1
  • proof
  • Remark 3.2
  • Example 5.1
  • Example 5.2