Stochastic control for Backward Stochastic Differential Equations with semi-Markov chain noises
Robert J. Elliott, Zhe Yang
TL;DR
The paper investigates discrete-time stochastic control where the driving noise arises from a semi-Markov chain, extending BSDE/SDE duality to non-Gaussian noise. It establishes existence and comparison results for BSDEs with semi-Markov noise and develops a Hamiltonian-based verification approach for optimal controls, without relying on PDE methods. By representing the cost through linear BSDEs and applying measurable selection, the authors show that the value function equals the essential supremum over admissible controls and characterize optimal strategies. The work provides a tractable, non-PDE framework for control problems under semi-Markov disturbances, with explicit adjoint constructions and stability results.
Abstract
In this paper, we extend the results of Elliott and Yang \cite{elliott3} and discuss the control of a stochastic process for which the driving noise is provided by a martingale associated with a semi-Markov Chain. An existence and a comparison theorem are obtained. In our discrete time setting, adjoint processes are provided by backward stochastic difference equations. Technical results from partial differential equation theory to establish a verification theorem are not required.