On the Value Function of Convex Bolza Problems Governed by Stochastic Difference Equations
Sebastián Álvarez, Julio Deride, Cristopher Hermosilla
TL;DR
This paper studies the value function of convex, non-anticipative stochastic Bolza problems in discrete time and connects the evolution of its subgradients to a stochastic discrete-time Hamiltonian system via a duality-based method. It derives a dual representation using Fenchel conjugates, establishes strong duality under standard assumptions, and formulates a discrete-time method of characteristics whereby subgradients correspond to backward Hamiltonian trajectories. The framework is then applied to linear-convex problems, showing existence of primal and dual optimizers and enabling a Hamiltonian characterization; the Linear-Quadratic case is explicitly worked out to yield concrete dynamics and transversality conditions. Collectively, the results provide a rigorous mechanism to analyze subgradient evolution and optimality conditions in stochastic discrete-time control with non-anticipativity, with practical applicability to LC and LQ problems.
Abstract
In this paper we study the value function of Bolza problems governed by stochastic difference equations, with particular emphasis on the convex non-anticipative case. Our goal is to provide some insights on the structure of the subdiferential of the value function. In particular, we establish a connection between the evolution of the subgradients of the value function and a stochastic difference equation of Hamiltonian type. This result can be seen as a transposition of the method of characteristics, introduced by Rockafellar and Wolenski in the 2000s, to the stochastic discrete-time setting. Similarly as done in the literature for the deterministic case, the analysis is based on a duality approach. For this reason we study first a dual representation for the value function in terms of the value function of a dual problem, which is a pseudo Bolza problem. The main difference with the deterministic case is that (due to the non-anticipativity) the symmetry between the Bolza problem and its dual is no longer valid. This in turn implies that ensuring the existence of minimizers for the Bolza problem (which is a key point for establishing the method of characteristics) is not as simple as in the deterministic case, and it should be addressed differently. To complete the exposition, we study the existence of minimizers for a particular class of Bolza problems governed by linear stochastic difference equations, the so-called linear-convex optimal control problems.