Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Optimal bounds for numerical approximations of infinite horizon problems based on dynamic programming approach

Javier de Frutos, Julia Novo

TL;DR

This work analyzes fully discrete semi-Lagrangian schemes for infinite-horizon optimal control governed by Hamilton-Jacobi-Bellman equations. By introducing a fully discrete cost functional that incorporates spatial interpolation and establishing a DP-based characterization, the authors prove that the discrete value function converges with first-order accuracy in both time and space under suitable regularity, addressing previously reported $O(k/h)$ bounds. They provide a nuanced error analysis: with regular controls (e.g., Lipschitz), the rate is $O(h+k)$, while weaker regularity or piecewise-constant controls yield slower but still convergent rates, including refined bounds under convexity assumptions. The results help reconcile theoretical bounds with observed numerical behavior and offer a framework potentially applicable to other HJB problems in optimal control.

Abstract

In this paper we get error bounds for fully discrete approximations of infinite horizon problems via the dynamic programming approach. It is well known that considering a time discretization with a positive step size $h$ an error bound of size $h$ can be proved for the difference between the value function (viscosity solution of the Hamilton-Jacobi-Bellman equation corresponding to the infinite horizon) and the value function of the discrete time problem. However, including also a spatial discretization based on elements of size $k$ an error bound of size $O(k/h)$ can be found in the literature for the error between the value functions of the continuous problem and the fully discrete problem. In this paper we revise the error bound of the fully discrete method and prove, under similar assumptions to those of the time discrete case, that the error of the fully discrete case is in fact $O(h+k)$ which gives first order in time and space for the method. This error bound matches the numerical experiments of many papers in the literature in which the behaviour $1/h$ from the bound $O(k/h)$ have not been observed.

Optimal bounds for numerical approximations of infinite horizon problems based on dynamic programming approach

TL;DR

This work analyzes fully discrete semi-Lagrangian schemes for infinite-horizon optimal control governed by Hamilton-Jacobi-Bellman equations. By introducing a fully discrete cost functional that incorporates spatial interpolation and establishing a DP-based characterization, the authors prove that the discrete value function converges with first-order accuracy in both time and space under suitable regularity, addressing previously reported bounds. They provide a nuanced error analysis: with regular controls (e.g., Lipschitz), the rate is , while weaker regularity or piecewise-constant controls yield slower but still convergent rates, including refined bounds under convexity assumptions. The results help reconcile theoretical bounds with observed numerical behavior and offer a framework potentially applicable to other HJB problems in optimal control.

Abstract

In this paper we get error bounds for fully discrete approximations of infinite horizon problems via the dynamic programming approach. It is well known that considering a time discretization with a positive step size an error bound of size can be proved for the difference between the value function (viscosity solution of the Hamilton-Jacobi-Bellman equation corresponding to the infinite horizon) and the value function of the discrete time problem. However, including also a spatial discretization based on elements of size an error bound of size can be found in the literature for the error between the value functions of the continuous problem and the fully discrete problem. In this paper we revise the error bound of the fully discrete method and prove, under similar assumptions to those of the time discrete case, that the error of the fully discrete case is in fact which gives first order in time and space for the method. This error bound matches the numerical experiments of many papers in the literature in which the behaviour from the bound have not been observed.
Paper Structure (5 sections, 11 theorems, 123 equations)

This paper contains 5 sections, 11 theorems, 123 equations.

Table of Contents

  1. Introduction
  2. Model problem and Preliminary results
  3. Optimal error bounds for the fully discrete approximations
  4. Error analysis assuming some regularity for the controls
  5. Error analysis arguing with piecewise constants controls

Key Result

Theorem 1

Let assumptions lip_f, infty_f, invariance, lip_g, cota_g, semi_con_f and semi_con_g hold and let $\lambda>\max(2L_g,L_f)$. Let $v$ and $v_h$ be the solutions of HJB and discrete_HJB, respectively. Then, there exists a constant $C\ge 0$, that can be bounded explicitly, such that the following bound

Theorems & Definitions (18)

  • Theorem 1
  • Theorem 2
  • Theorem 3
  • Proof 1
  • Lemma 1
  • Proof 2
  • Lemma 2
  • Proof 3
  • Theorem 4
  • Proof 4
  • ...and 8 more