Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

An infinite horizon sufficient stochastic maximum principle for regime switching diffusions and applications

Kai Ding, Xun Li, Siyu Lv, Xin Zhang

TL;DR

This work develops an infinite-horizon discounted stochastic maximum principle for regime-switching diffusions by first proving global well-posedness and asymptotic behavior for infinite-horizon SDEs/BSDEs with Markov chains under a discount factor $r>0$. Using a dual approach and convexity of the Hamiltonian, it establishes a sufficient SMP that does not require a transversality condition thanks to asymptotic properties. The authors apply the principle to a one-dimensional production-planning problem with regime-switching, deriving an explicit feedback control via an algebraic Riccati equation and a linear equation, and proving the existence and uniqueness of a nonnegative Riccati solution. Numerical experiments illustrate how regime switching and model parameters affect the value function and optimal control, highlighting monotonicity and robustness features of the framework.

Abstract

This paper is concerned with a discounted stochastic optimal control problem for regime switching diffusion in an infinite horizon. First, as a preliminary with particular interests in its own right, the global well-posedness of infinite horizon forward and backward stochastic differential equations with Markov chains and the asymptotic property of their solutions when time goes to infinity are obtained. Then, a sufficient stochastic maximum principle for optimal controls is established via a dual method under certain convexity condition of the Hamiltonian. As an application of our maximum principle, a linear quadratic production planning problem is solved with an explicit feedback optimal production rate. The existence and uniqueness of a non-negative solution to the associated algebraic Riccati equation are proved. Numerical experiments are reported to illustrate the theoretical results, especially, the monotonicity of the value function on various model parameters.

An infinite horizon sufficient stochastic maximum principle for regime switching diffusions and applications

TL;DR

This work develops an infinite-horizon discounted stochastic maximum principle for regime-switching diffusions by first proving global well-posedness and asymptotic behavior for infinite-horizon SDEs/BSDEs with Markov chains under a discount factor . Using a dual approach and convexity of the Hamiltonian, it establishes a sufficient SMP that does not require a transversality condition thanks to asymptotic properties. The authors apply the principle to a one-dimensional production-planning problem with regime-switching, deriving an explicit feedback control via an algebraic Riccati equation and a linear equation, and proving the existence and uniqueness of a nonnegative Riccati solution. Numerical experiments illustrate how regime switching and model parameters affect the value function and optimal control, highlighting monotonicity and robustness features of the framework.

Abstract

This paper is concerned with a discounted stochastic optimal control problem for regime switching diffusion in an infinite horizon. First, as a preliminary with particular interests in its own right, the global well-posedness of infinite horizon forward and backward stochastic differential equations with Markov chains and the asymptotic property of their solutions when time goes to infinity are obtained. Then, a sufficient stochastic maximum principle for optimal controls is established via a dual method under certain convexity condition of the Hamiltonian. As an application of our maximum principle, a linear quadratic production planning problem is solved with an explicit feedback optimal production rate. The existence and uniqueness of a non-negative solution to the associated algebraic Riccati equation are proved. Numerical experiments are reported to illustrate the theoretical results, especially, the monotonicity of the value function on various model parameters.

Paper Structure

This paper contains 15 sections, 9 theorems, 122 equations, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Problem formulation
  3. Infinite horizon (B)SDEs with Markov chains
  4. Infinite horizon SDEs with Markov chains
  5. Infinite horizon BSDEs with Markov chains
  6. The associated stochastic Hamiltonian system
  7. Sufficient stochastic maximum principle
  8. A production planning problem
  9. Model and solution
  10. Numerical experiments
  11. A basic case
  12. Sensitivity analysis
  13. Concluding remarks
  14. Proof of Lemma \ref{['lem:Y']}
  15. Proof of Lemma \ref{['lem:Riccatisolvable']}

Key Result

Theorem 3.1

Let Assumptions (H1)-(H3) hold. For any $u\in L_{\mathbb{F}}^{2,K}(0,\infty;\mathbb{R}^k)$, the SDE (FSDE) admits a unique strong solution $X\in L_{\mathbb{F}}^{2,K}(0,\infty;\mathbb{R}^n)$ with $K<(\kappa_1-\frac{1}{2}\kappa_{\sigma}^2)\wedge 0$. Moreover, for any $\varepsilon>0$, we have and

Figures (3)

  • Figure 1: Value function $v(x,i)$.
  • Figure 2: Simulation of $X_t^*$ and $u_t^*$.
  • Figure 3: Dependency of value functions on model parameters.

Theorems & Definitions (22)

  • Theorem 3.1
  • proof
  • Definition 3.2
  • Lemma 3.3
  • proof
  • Lemma 3.4
  • proof
  • Theorem 3.5
  • proof
  • Corollary 3.6
  • ...and 12 more