Ergodic Risk Sensitive Control of Diffusions under a General Structural Hypothesis
Sumith Reddy Anugu, Guodong Pang
TL;DR
This work develops a general structural framework for ergodic risk-sensitive control of diffusions, where the state space is partitioned into two regions with distinct stability/cost properties. It introduces an inf-compact perturbation to the running cost and leverages a variational representation of exponential functionals to reformulate the ERSC problem in terms of an extended diffusion with an auxiliary control, enabling a TP-ZS game analysis to justify interchanging limits and sup/inf operations. The main contributions are the well-posedness of the multiplicative HJB equation, the complete characterization of optimal stationary Markov controls, and a rigorous limiting procedure proving that the ERSC value can be attained by stationary Markov controls under the general structural hypothesis. The results extend prior near-monotone and blanket stability theories by providing a unified approach applicable to diffusion models arising in complex networks and Halfin-Whitt regimes, along with detailed stochastic representations and ground-diffusion stability properties.
Abstract
We study the infinite-horizon average (ergodic) risk sensitive control problem for diffusion processes under a general structural hypothesis: there is a partition of state space into two subsets, where the controlled diffusion process satisfies a Foster-Lyapunov type drift condition in one subset, under any stationary Markov control, while the near-monotonicity condition is satisfied with the running cost function being inf-compact in its complement. Under these conditions, we completely characterize the optimal stationary Markov controls. To prove this, we consider an inf-compact perturbation to the running cost over the entire space such that the resulting ergodic risk sensitive control problem is well-defined and then use the corresponding existing results. The heart of the analysis lies in exploiting the variational formula of exponential functionals of Brownian motion and applying it to the objective exponential cost function of the controlled diffusion. This representation facilitates us to view the risk sensitive cost for any stationary Markov control as the optimal value of a control problem of an extended diffusion involving a new auxiliary control where the optimal criterion is to maximize the associated long-run average cost criterion that is a difference of the original running cost and an extra term that is quadratic in the auxiliary control. The main difficulty in using this approach lies in the fact that tightness of mean empirical measures of the extended diffusion is not a priori implied by the analogous tightness property of the original diffusion. We overcome this by establishing a priori estimates for the extended diffusion associated with the nearly optimal auxiliary controls.