Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning to steer with Brownian noise

Stefan Ankirchner, Sören Christensen, Jan Kallsen, Philip Le Borne, Stefan Perko

Abstract

This paper considers an ergodic version of the bounded velocity follower problem, assuming that the decision maker lacks knowledge of the underlying system parameters and must learn them while simultaneously controlling. We propose algorithms based on moving empirical averages and develop a framework for integrating statistical methods with stochastic control theory. Our primary result is a logarithmic expected regret rate. To achieve this, we conduct a rigorous analysis of the ergodic convergence rates of the underlying processes and the risks of the considered estimators.

Learning to steer with Brownian noise

Abstract

This paper considers an ergodic version of the bounded velocity follower problem, assuming that the decision maker lacks knowledge of the underlying system parameters and must learn them while simultaneously controlling. We propose algorithms based on moving empirical averages and develop a framework for integrating statistical methods with stochastic control theory. Our primary result is a logarithmic expected regret rate. To achieve this, we conduct a rigorous analysis of the ergodic convergence rates of the underlying processes and the risks of the considered estimators.
Paper Structure (10 sections, 11 theorems, 94 equations)

This paper contains 10 sections, 11 theorems, 94 equations.

Table of Contents

  1. Introduction
  2. Problem formulation
  3. Structure of the paper and main contributions
  4. Related Literature
  5. The ergodic version of the bounded velocity follower problem
  6. Learning the optimal control
  7. Properties of Brownian motion with broken drift
  8. Proof of the regret rates
  9. Proof of Theorem \ref{['thm 1']}
  10. Proof of Theorem \ref{['regret']}

Key Result

Theorem 2.1

Let $\delta$ be defined as in defi delta and let $x \in \mathbb{R}$. Then for any $x_0 \in \mathbb{R}$ there exists a unique strong solution of the SDE $dX_t = b^*(X_t) dt + dW_t$ with initial condition $X_0 = x_0$. Moreover, the Markovian control $b^*_t:=b^*(X_t)$, $t \in [0, \infty)$, is optimal in $U$ and the value function satisfies

Theorems & Definitions (24)

  • Theorem 2.1
  • proof
  • Theorem 3.1
  • proof
  • Theorem 3.2
  • proof
  • Lemma 4.1
  • proof
  • Lemma 4.2
  • proof
  • ...and 14 more