Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning to reflect: A unifying approach for data-driven stochastic control strategies

Sören Christensen, Claudia Strauch, Lukas Trottner

Abstract

Stochastic optimal control problems have a long tradition in applied probability, with the questions addressed being of high relevance in a multitude of fields. Even though theoretical solutions are well understood in many scenarios, their practicability suffers from the assumption of known dynamics of the underlying stochastic process, raising the statistical challenge of developing purely data-driven strategies. For the mathematically separated classes of continuous diffusion processes and Lévy processes, we show that developing efficient strategies for related singular stochastic control problems can essentially be reduced to finding rate-optimal estimators with respect to the sup-norm risk of objects associated to the invariant distribution of ergodic processes which determine the theoretical solution of the control problem. From a statistical perspective, we exploit the exponential $β$-mixing property as the common factor of both scenarios to drive the convergence analysis, indicating that relying on general stability properties of Markov processes is a sufficiently powerful and flexible approach to treat complex applications requiring statistical methods. We show moreover that in the Lévy case $-$ even though per se jump processes are more difficult to handle both in statistics and control theory $-$ a fully data-driven strategy with regret of significantly better order than in the diffusion case can be constructed.

Learning to reflect: A unifying approach for data-driven stochastic control strategies

Abstract

Stochastic optimal control problems have a long tradition in applied probability, with the questions addressed being of high relevance in a multitude of fields. Even though theoretical solutions are well understood in many scenarios, their practicability suffers from the assumption of known dynamics of the underlying stochastic process, raising the statistical challenge of developing purely data-driven strategies. For the mathematically separated classes of continuous diffusion processes and Lévy processes, we show that developing efficient strategies for related singular stochastic control problems can essentially be reduced to finding rate-optimal estimators with respect to the sup-norm risk of objects associated to the invariant distribution of ergodic processes which determine the theoretical solution of the control problem. From a statistical perspective, we exploit the exponential -mixing property as the common factor of both scenarios to drive the convergence analysis, indicating that relying on general stability properties of Markov processes is a sufficiently powerful and flexible approach to treat complex applications requiring statistical methods. We show moreover that in the Lévy case even though per se jump processes are more difficult to handle both in statistics and control theory a fully data-driven strategy with regret of significantly better order than in the diffusion case can be constructed.

Paper Structure

This paper contains 10 sections, 5 theorems, 41 equations.

Table of Contents

  1. Introduction
  2. The motivating control problems
  3. Nonparametric analysis: Controlling the $\mathbf{sup}$-norm risk of Markovian functionals
  4. Basic notions and results
  5. Application in the scalar setting
  6. Organization of the paper
  7. Data-driven singular controls for diffusions on the real line
  8. Estimating the stationary density of ergodic diffusion processes
  9. Application
  10. Solution for known drift

Key Result

Proposition \oldthetheorem

Let $\mathcal{G}$ be a countable class of bounded real-valued functions $g$ satisfying $\mu(g) = 0$, and define Suppose that $\mathbf{X}$ is stationary with invariant distribution $\mu$ and exponentially $\beta$-mixing, and let $m_t \in (0, t\slash 4]$. Then, there exist $\tau \in [m_t, 2m_t]$ and constants $C_1,C_2,c_1,c_2>0$ such that, for any $1\le p<\infty$, Here, for $f,g \in \mathcal{G}$,

Theorems & Definitions (8)

  • Proposition \oldthetheorem: Theorem 3.2 in dexheimer20
  • Theorem \oldthetheorem: concentration of invariant density estimators
  • proof
  • Lemma \oldthetheorem
  • proof
  • Corollary \oldthetheorem
  • proof
  • Lemma \oldthetheorem