Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Policy iteration for the deterministic control problems -- a viscosity approach

Wenpin Tang, Hung Vinh Tran, Yuming Paul Zhang

TL;DR

It is proved that PI for the semi-discrete scheme converges exponentially fast, and a bound is provided on the error induced by the Semi-Discrete Scheme.

Abstract

This paper is concerned with the convergence rate of policy iteration for (deterministic) optimal control problems in continuous time. To overcome the problem of ill-posedness due to lack of regularity, we consider a semi-discrete scheme by adding a viscosity term via finite differences in space. We prove that PI for the semi-discrete scheme converges exponentially fast, and provide a bound on the error induced by the semi-discrete scheme. We also consider the discrete space-time scheme, where both space and time are discretized. Convergence rate of PI and the discretization error are studied.

Policy iteration for the deterministic control problems -- a viscosity approach

TL;DR

It is proved that PI for the semi-discrete scheme converges exponentially fast, and a bound is provided on the error induced by the Semi-Discrete Scheme.

Abstract

This paper is concerned with the convergence rate of policy iteration for (deterministic) optimal control problems in continuous time. To overcome the problem of ill-posedness due to lack of regularity, we consider a semi-discrete scheme by adding a viscosity term via finite differences in space. We prove that PI for the semi-discrete scheme converges exponentially fast, and provide a bound on the error induced by the semi-discrete scheme. We also consider the discrete space-time scheme, where both space and time are discretized. Convergence rate of PI and the discretization error are studied.
Paper Structure (15 sections, 17 theorems, 147 equations, 1 figure)

This paper contains 15 sections, 17 theorems, 147 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Setup and preliminary results
  3. Semi-discrete schemes
  4. Discrete space-time schemes
  5. Analysis of semi-discrete schemes
  6. Convergence of PI
  7. Convergence of $v^h$ as $h\to 0$
  8. Almost everywhere convergence of the policy
  9. Analysis of discrete space-time schemes
  10. Convergence of PI
  11. Convergence of $V^{\tau,h}$ as $\tau, h \to 0$.
  12. Almost everywhere convergence of the policy
  13. Generalization: a PDE perspective
  14. Numerical experiments
  15. Conclusion

Key Result

Lemma 2.1

Let $v_0^h$ and $\tilde{v}_0^h$ be, respectively, a bounded continuous super- and sub- solution to p.1 with $n=0$, and satisfy $\tilde{v}_0^h\leq {v}_0^h$ at $t=T$. Then $\tilde{v}_0^h\leq {v}_0^h$ in $[0,T]\times{\mathbb{R}}^d$. Here by a supersolution (resp. subsolution), we mean that it satisfies

Figures (1)

  • Figure 1: Convergence of PI \ref{['4.1']}--\ref{["4.1'"]} for $f(t,x,a) = a$, $c(t,x,a) = \frac{1}{2} a^2$ and $q \equiv 0$, with $A = [-2, 2]$, $\tau = 0.025$, $h = 0.1$ and $N = 2$.

Theorems & Definitions (30)

  • Lemma 2.1
  • Proposition 2.2
  • proof
  • Proposition 2.3
  • proof
  • Lemma 2.4
  • Proposition 2.5
  • Theorem 3.1
  • proof
  • Remark 3.1
  • ...and 20 more